RLiterate is a tool for authoring documents. You can think of it as a programmer's version of a word processor. This book describes RLiterate and also includes the complete implementation of the program presented in small pieces.
Screenshot of RLiterate editing itself.
This chapter gives an overview of what types of document you can create with RLiterate and presents the main features of the program.
RLiterate documents have pages organized in a hierarchy. Pages have a title and paragraphs. Paragraphs can be of different types. The different paragraph types is what make RLiterate documents special. Code paragraphs for example enable literate programming by allowing chunks of code to be defined and then be automatically assembled into the final source file. Text paragraphs are used for writing prose. RLiterate documents can also be exported to different formats for display in different mediums.
The main GUI consists of the table of contents and the workspace.
The main GUI window.
The table of contents shows the outline of the document, and from it, pages can be opened in the workspace. Each page is drawn with a border in the workspace.
TODO: Explain how code paragraphs enable literate programming.
RLiterate is not only a tool for authoring documents, but also a tool for reading documents. The following features support that.
Hoisting a page in the table of contents allows you to focus on a subset of the document.
Page 1 hoisted.
Openining a page and all immediate children allows you to read a subset of the document breath first. It's like reading only the first section of each chapter in an entire book.
Demo page with its children opened. Notice that Sub page is not an immediate child and therefore not shown.
This chapter gives practical advice for using RLiterate on your computer.
RLiterate is currently a prototype. Experimentation is encouraged, but it may have serious bugs.
The source code for RLiterate is in a Git repository hosted on Github:
https://github.com/rickardlindberg/rliterate
Assuming you have Git installed, you can clone the repository like this:
git clone https://github.com/rickardlindberg/rliterate.git
Before you can run RLiterate, you need Python with the following libraries installed:
Here is how they can be installed on a Fedora system:
dnf install python wxPython python2-pygments
Once they are installed, you can run RLiterate like this:
python rliterate.py rliterate.rliterate
This will open the RLiterate document. Enter a different filename as last parameter to create a new document.
When *.rliterate files are version controlled, the textual diff is hard to read. This problem can be sovled in Git by defining a textconv command that converts the *.rliterate file to text that is suitable for diffing. The --diff option to RLiterate outputs a file in a diff friendly format.
First associate *.rliterate files with rliterate using a Git attributes file. For example in $HOME/.config/git/attributes:
*.rliterate diff=rliterate
Then define the textconv command in your git config ($HOME/.gitconfig):
[diff "rliterate"] textconv=bash -c 'python $RLITERATE_ROOT/rliterate.py "$0" --diff'
This chapter explains why RLiterate exists and how it came about.
I started to think about what would become RLiterate when I read the paper Active Essays on the Web. In it they talk about embedding code in documents that the reader can interact with. They also mention Literate programming as having a related goal.
At the time I was working on a program that I thought would be nice to write in this way. I wanted to write an article about the program and have the code for the program embedded in the article. I could have used a literate programming tool for this, but the interactive aspect of active essays made me think that a tool would be much more powerful if the document could be edited "live", similar to WYSIWYG editors. Literate programming tools I were aware of worked by editing plain text files with a special syntax for code and documentation blocks, thus lacking the interactive aspect.
So I decided to build a prototype to learn what such a program might be like.
First I came up with a document model where pages were organized in a hierarchy. Each page had paragraphs that could be of different types. This idea was stolen from Smallest Federated Wiki. The code paragraph would allow for literate programming, while the text paragraph would be for prose. I also envisioned other paragraph types that would allow for more interaction. Perhaps one paragraph type could be Graphviz code, and when edited, a generated graph would appear instead of the code.
After coming up with a document model, I implement a GUI that would allow editing such documents. This GUI had to be first class as it would be the primary way author (and read) documents.
The first version of the GUI was not first class though. I started to experiment with a workspace for showing pages.
First GUI prototype of workspace.
Then I contintued with a table of contents. At this point it was based on a tree control present in wxPython.
First GUI prototype of table of contents. Workspace pages also have titles.
All the data so far had been static and hard coded. I started to flesh out the document model and implement GUI controls for manipulating the document. Here is the add button on pages.
Add button that creates the factory paragraph.
Drag and drop was implemented fairly early. I wanted good visual feedback. This was part of the first class idea.
Drag and drop with red divider line indicating drop point.
Drag and drop with good visual feedback was hard to achieve with the tree control from wxWidgets, so at this point I rewrote the table of contents widget as a custom widget.
Custom table of contents that allows drag and drop and custom rendering.
I added more operations for adding and deleting pages.
Context menu that allows adding and deleting pages.
Then finally I added a second paragraph type: the code paragraph type that would enable literate programming.
Code paragraphs showing literate programming in action.
At this point I had all the functionality in place for writing documents and embedding code in them. I imported all the code into an RLiterate document (previously it was written as a single Python file) and started extracting pieces and adding prose to explain the program. This was a tremendously rewarding experience. RLiterate was now bootstrapped.
As I continued to improve the RLiterate document, I noticed features I was lacking. I added them to the program and iterated.
To me, the most central idea in literate programming is that we should write programs for other humans. Only secondary for the machine. I find code easier to understand if I can understand it in isolated pieces. One example where it is difficult to isolate a piece without literate programming is test code. Usually the test code is located in one file, and the implementation in another. To fully understand the piece it is useful to both read the tests and the implementation. To do this we have to find this related information in two unrelated files. I wrote about this problem in 2013 in Related things are not kept together. Literate programming allows us to present the test and the implementation in the same place, yet have the different pieces written to different files. The compiler might require that they are in separate files, but with literate programming, we care first about the other human that will read our code, and only second about the compiler.
Another argument for literate programming is to express the "why". Why is this code here? Timothy Daly talks about it in his talk Literate Programming in the Large. He also argues that programmers must change the mindset from wring a program to writing a book. Some more can be read in Example of Literate Programming in HTML.
Some more resources about literate programming:
When researching ideas for RLiterate, I stumbled across ProjecturED. It is similar to RLiterate in the sense that it is an editor for more structured documents. Not just text. The most interesting aspect for me was that a variable name exists in one place, but can be rendered in multiple. So a rename is really simple. With RLiterate, you have to do a search and replace. But with ProjecturED you just change the name and it replicates everywhere. This is an attractive feature and is made possible by the different document model.
This chapter gives a complete description of the implementation of RLiterate presented in small pieces.
RLiterate is implemented in Python. The implementation consists of three files: one for the program, one for the test code, and a Makefile for various operations.
# This file is extracted from rliterate.rliterate. # DO NOT EDIT MANUALLY! <<imports>> <<constants>> <<base base classes>> <<base classes>> <<classes>> <<functions>> if __name__ == "__main__": <<entry point>>
# This file is extracted from rliterate.rliterate. # DO NOT EDIT MANUALLY! <<imports>> <<fixtures>> <<test cases>>
# This file is extracted from rliterate.rliterate. # DO NOT EDIT MANUALLY! <<rules>>
The remainder of this chapter will fill in the sections in those files to create the complete implementation.
An RLiterate document is a JSON object (hereafter called the document dict) stored in a file. The Document class provides a friendly interface for working with RLiterate documents.
Observable so that others can listen for changes.The Document class is initialized with a documetn dict (or None to create a new document):
class Document(Observable): def __init__(self, document_dict=None): Observable.__init__(self) self._load(document_dict) <<Document>>
The default constructor does not know anything about files. It just creates an in-memory document. The from_file constructor reads a document from file and also writes it back to file when it changes:
@classmethod def from_file(cls, path): if os.path.exists(path): document = cls(load_json_from_file(path)) else: document = cls() document.listen(lambda: write_json_to_file( path, document.read_only_document_dict ) ) return document
Loading is the same in both cases and involves
def _load(self, document_dict): self._history = History( DocumentDictWrapper( self._convert_to_latest( self._empty_page() if document_dict is None else document_dict ) ), size=UNDO_BUFFER_SIZE )
UNDO_BUFFER_SIZE = 10
The current document dict can be accessed via the read_only_document_dict property. But it should be considered read only. For undo/redo to work properly, all edits must go through modify.
@property def read_only_document_dict(self): return self._history.value
There are only three ways in which a document should be modified:
modify method@contextlib.contextmanager def modify(self, name): with self.notify(): with self._history.new_value(name) as value: yield value def get_undo_operation(self): def undo(): with self.notify(): self._history.back() if self._history.can_back(): return (self._history.back_name(), undo) def get_redo_operation(self): def redo(): with self.notify(): self._history.forward() if self._history.can_forward(): return (self._history.forward_name(), redo)
The document dict wrapper is a helper for working with a document dict. It mainly maintains caches.
A document dict has the following structure:
{ "root_page": { "id": "106af6f8665c45e8ab751993a6abc876", "title": "Root", "paragraphs": [ { "id": "8f0a9f84821540e89d7c9ca93ed0fbe7", "type": "text", "fragments": [ {"text": "This ", "type": "text" }, {"text": "is", "type": "strong" }, {"text": "cool", "type": "emphasis" } ] }, ... ], "children": [ ... ] }, "variables": { ... } }
The root object represents a document. It has a root page and variables. The root page has a unique id, a title, paragraphs, and child pages. Paragraphs also have unique ids and different attributes depending on type. (The text paragraph has a list of text fragments with different styles.)
class DocumentDictWrapper(dict): def __init__(self, document_dict): dict.__init__(self, document_dict) self._pages = {} self._parent_pages = {} self._paragraphs = {} self._cache_page(self["root_page"]) def _cache_page(self, page, parent_page=None): self._pages[page["id"]] = page self._parent_pages[page["id"]] = parent_page for paragraph in page["paragraphs"]: self._paragraphs[paragraph["id"]] = paragraph for child in page["children"]: self._cache_page(child, page) def add_page_dict(self, page_dict, parent_id=None): page_dict = copy.deepcopy(page_dict) parent_page = self._pages[parent_id] parent_page["children"].append(page_dict) self._pages[page_dict["id"]] = page_dict self._parent_pages[page_dict["id"]] = parent_page def get_page_dict(self, page_id=None): if page_id is None: page_id = self["root_page"]["id"] return self._pages.get(page_id, None) def get_parent_page_dict(self, page_id): return self._parent_pages.get(page_id, None) def delete_page_dict(self, page_id): if page_id == self["root_page"]["id"]: return page = self._pages[page_id] parent_page = self._parent_pages[page_id] index = index_with_id(parent_page["children"], page_id) parent_page["children"].pop(index) self._pages.pop(page_id) self._parent_pages.pop(page_id) for child in reversed(page["children"]): parent_page["children"].insert(index, child) self._parent_pages[child["id"]] = parent_page def update_page_dict(self, page_id, data): self._pages[page_id].update(copy.deepcopy(data)) def move_page_dict(self, page_id, parent_page_id, before_page_id): if page_id == before_page_id: return parent = self._pages[parent_page_id] while parent is not None: if parent["id"] == page_id: return parent = self._parent_pages[parent["id"]] parent = self._parent_pages[page_id] page = parent["children"].pop(index_with_id(parent["children"], page_id)) new_parent = self._pages[parent_page_id] self._parent_pages[page_id] = new_parent if before_page_id is None: new_parent["children"].append(page) else: new_parent["children"].insert( index_with_id(new_parent["children"], before_page_id), page ) def paragraph_dict_iterator(self): return self._paragraphs.values() def add_paragraph_dict(self, paragraph_dict, page_id, before_id): paragraph_dict = copy.deepcopy(paragraph_dict) paragraphs = self._pages[page_id]["paragraphs"] if before_id is None: paragraphs.append(paragraph_dict) else: paragraphs.insert(index_with_id(paragraphs, before_id), paragraph_dict) self._paragraphs[paragraph_dict["id"]] = paragraph_dict def delete_paragraph_dict(self, page_id, paragraph_id): paragraphs = self._pages[page_id]["paragraphs"] paragraphs.pop(index_with_id(paragraphs, paragraph_id)) return self._paragraphs.pop(paragraph_id) def move_paragraph_dict(self, page_id, paragraph_id, target_page, before_paragraph): if (page_id == target_page and paragraph_id == before_paragraph): return self.add_paragraph_dict( self.delete_paragraph_dict(page_id, paragraph_id), target_page, before_paragraph ) def update_paragraph_dict(self, paragraph_id, data): self._paragraphs[paragraph_id].update(copy.deepcopy(data))
def index_with_id(items, item_id): for index, item in enumerate(items): if item["id"] == item_id: return index
def add_page(self, title="New page", parent_id=None): with self.modify("Add page") as document_dict: document_dict.add_page_dict(self._empty_page(), parent_id=parent_id) def _empty_page(self): return { "id": genid(), "title": "New page...", "children": [], "paragraphs": [], } def iter_pages(self): def iter_pages(page): yield page for child in page.children: for sub_page in iter_pages(child): yield sub_page return iter_pages(self.get_page())
def get_page(self, page_id=None): page_dict = self.read_only_document_dict.get_page_dict(page_id) if page_dict is None: return None return Page(self, page_dict) def get_parent_page(self, page_id): page_dict = self.read_only_document_dict.get_parent_page_dict(page_id) if page_dict is None: return None return Page(self, page_dict)
class Page(object): def __init__(self, document, page_dict): self._document = document self._page_dict = page_dict @property def parent(self): return self._document.get_parent_page(self.id) @property def full_title(self): return " / ".join(page.title for page in self.chain) @property def chain(self): result = [] page = self while page is not None: result.insert(0, page) page = page.parent return result def iter_code_fragments(self): for paragraph in self.paragraphs: for fragment in paragraph.iter_code_fragments(): yield fragment def iter_text_fragments(self): for paragraph in self.paragraphs: for fragment in paragraph.iter_text_fragments(): yield fragment @property def id(self): return self._page_dict["id"] @property def title(self): return self._page_dict["title"] def set_title(self, title): with self._document.modify("Change title") as document_dict: document_dict.update_page_dict(self.id, {"title": title}) @property def paragraphs(self): return [ Paragraph.create( self._document, self, paragraph_dict, next_paragraph_dict["id"] if next_paragraph_dict is not None else None ) for paragraph_dict, next_paragraph_dict in zip( self._page_dict["paragraphs"], self._page_dict["paragraphs"][1:]+[None] ) ] @property def children(self): return [ Page(self._document, child_dict) for child_dict in self._page_dict["children"] ] def delete(self): with self._document.modify("Delete page") as document_dict: document_dict.delete_page_dict(self.id) def move(self, parent_page_id, before_page_id): with self._document.modify("Move page") as document_dict: document_dict.move_page_dict(self.id, parent_page_id, before_page_id)
def add_paragraph(self, page_id, before_id=None, paragraph_dict={"type": "factory"}): with self.modify("Add paragraph") as document_dict: document_dict.add_paragraph_dict( dict(paragraph_dict, id=genid()), page_id, before_id=before_id ) def get_paragraph(self, page_id, paragraph_id): for paragraph in self.get_page(page_id).paragraphs: if paragraph.id == paragraph_id: return paragraph
class Paragraph(object): @staticmethod def create(document, page, paragraph_dict, next_id): return { "text": TextParagraph, "quote": QuoteParagraph, "list": ListParagraph, "code": CodeParagraph, "image": ImageParagraph, "expanded_code": ExpandedCodeParagraph, }.get(paragraph_dict["type"], Paragraph)(document, page, paragraph_dict, next_id) def __init__(self, document, page, paragraph_dict, next_id): self._document = document self._page = page self._paragraph_dict = paragraph_dict self._next_id = next_id @property def id(self): return self._paragraph_dict["id"] @property def next_id(self): return self._next_id @property def type(self): return self._paragraph_dict["type"] @contextlib.contextmanager def multi_update(self): with self._document.modify("Edit paragraph"): yield def update(self, data): with self._document.modify("Edit paragraph") as document_dict: document_dict.update_paragraph_dict(self.id, data) def delete(self): with self._document.modify("Delete paragraph") as document_dict: document_dict.delete_paragraph_dict(self._page.id, self.id) def move(self, target_page, before_paragraph): with self._document.modify("Move paragraph") as document_dict: document_dict.move_paragraph_dict(self._page.id, self.id, target_page, before_paragraph) def duplicate(self): with self._document.modify("Duplicate paragraph") as document_dict: document_dict.add_paragraph_dict( dict(copy.deepcopy(self._paragraph_dict), id=genid()), page_id=self._page.id, before_id=self.next_id ) @property def filename(self): return "paragraph.txt" def iter_code_fragments(self): return iter([]) def iter_text_fragments(self): return iter([])
class TextParagraph(Paragraph): @property def fragments(self): return TextFragment.create_list(self._document, self._paragraph_dict["fragments"]) @property def tokens(self): return [x.token for x in self.fragments] def get_text_index(self, index): return self._text_version.get_selection(index)[0] @property def text_version(self): return self._text_version.text @property def _text_version(self): text_version = TextVersion() for fragment in self.fragments: fragment.fill_text_version(text_version) return text_version @text_version.setter def text_version(self, value): self.update({"fragments": text_to_fragments(value)}) def iter_text_fragments(self): return iter(self.fragments)
def fragments_to_text(fragments): text_version = TextVersion() for fragment in fragments: fragment.fill_text_version(text_version) return text_version.text def text_to_fragments(text): return TextParser().parse(text)
class TextParser(object): SPACE_RE = re.compile(r"\s+") PATTERNS = [ ( re.compile(r"\*\*(.+?)\*\*", flags=re.DOTALL), lambda parser, match: { "type": "strong", "text": match.group(1), } ), ( re.compile(r"\*(.+?)\*", flags=re.DOTALL), lambda parser, match: { "type": "emphasis", "text": match.group(1), } ), ( re.compile(r"``(.+?)``", flags=re.DOTALL), lambda parser, match: { "type": "variable", "id": match.group(1), } ), ( re.compile(r"`(.+?)`", flags=re.DOTALL), lambda parser, match: { "type": "code", "text": match.group(1), } ), ( re.compile(r"\[\[(.+?)(:(.+?))?\]\]", flags=re.DOTALL), lambda parser, match: { "type": "reference", "text": match.group(3), "page_id": match.group(1), } ), ( re.compile(r"\[(.*?)\]\((.+?)\)", flags=re.DOTALL), lambda parser, match: { "type": "link", "text": match.group(1), "url": match.group(2), } ), ] def parse(self, text): text = self._normalise_space(text) fragments = [] partial = "" while text: result = self._get_special_fragment(text) if result is None: partial += text[0] text = text[1:] else: match, fragment = result if partial: fragments.append({"type": "text", "text": partial}) partial = "" fragments.append(fragment) text = text[match.end(0):] if partial: fragments.append({"type": "text", "text": partial}) return fragments def _normalise_space(self, text): return self.SPACE_RE.sub(" ", text).strip() def _get_special_fragment(self, text): for pattern, fn in self.PATTERNS: match = pattern.match(text) if match: return match, fn(self, match)
class QuoteParagraph(TextParagraph): pass
class ListParagraph(Paragraph): @property def child_type(self): return self._paragraph_dict["child_type"] @property def children(self): return [ListItem(self._document, x) for x in self._paragraph_dict["children"]] def get_text_index(self, list_and_fragment_index): return self._text_version.get_selection(list_and_fragment_index)[0] @property def text_version(self): return self._text_version.text @property def _text_version(self): def list_item_to_text(text_version, child_type, item, indent=0, index=0): text_version.add(" "*indent) if child_type == "ordered": text_version.add("{}. ".format(index+1)) else: text_version.add("* ") for fragment in item.fragments: fragment.fill_text_version(text_version) text_version.add("\n") for index, child in enumerate(item.children): with text_version.index(index): list_item_to_text(text_version, item.child_type, child, index=index, indent=indent+1) text_version = TextVersion() for index, child in enumerate(self.children): with text_version.index(index): list_item_to_text(text_version, self.child_type, child, index=index) return text_version @text_version.setter def text_version(self, value): child_type, children = LegacyListParser(value).parse_items() self.update({ "child_type": child_type, "children": children }) def iter_text_fragments(self): for item in self.children: for fragment in item.iter_text_fragments(): yield fragment
class ListItem(object): def __init__(self, document, item_dict): self._document = document self._item_dict = item_dict @property def fragments(self): return TextFragment.create_list(self._document, self._item_dict["fragments"]) @property def child_type(self): return self._item_dict["child_type"] @property def children(self): return [ListItem(self._document, x) for x in self._item_dict["children"]] @property def tokens(self): return [x.token for x in self.fragments] def iter_text_fragments(self): for fragment in self.fragments: yield fragment for child in self.children: for fragment in child.iter_text_fragments(): yield fragment
class CodeParagraph(Paragraph): <<CodeParagraph>>
@property def path(self): return Path( [x for x in self._paragraph_dict["filepath"] if x], [x for x in self._paragraph_dict["chunkpath"] if x] ) @path.setter def path(self, path): self.update({ "filepath": copy.deepcopy(path.filepath), "chunkpath": copy.deepcopy(path.chunkpath), })
@property def filename(self): return self.path.filename
class Path(object): @classmethod def from_text_version(cls, text): try: filepath_text, chunkpath_text = text.split(" // ", 1) except: filepath_text = text chunkpath_text = "" return cls( filepath_text.split("/") if filepath_text else [], chunkpath_text.split("/") if chunkpath_text else [], ) @property def text_version(self): if self.has_both(): sep = " // " else: sep = "" return "{}{}{}".format( "/".join(self.filepath), sep, "/".join(self.chunkpath) ) @property def text_start(self): return self.text_end - len(self.last) @property def text_end(self): return len(self.text_version) def extend_chunk(self, chunk): return Path( copy.deepcopy(self.filepath), copy.deepcopy(self.chunkpath)+copy.deepcopy(chunk) ) @property def filename(self): return self.filepath[-1] if self.filepath else "" @property def last(self): if len(self.chunkpath) > 0: return self.chunkpath[-1] elif len(self.filepath) > 0: return self.filepath[-1] else: return "" @property def is_empty(self): return self.length == 0 @property def length(self): return len(self.chunkpath) + len(self.filepath) def __init__(self, filepath, chunkpath): self.filepath = filepath self.chunkpath = chunkpath def is_prefix(self, other): if len(self.chunkpath) > 0: return self.filepath == other.filepath and self.chunkpath == other.chunkpath[:len(self.chunkpath)] else: return self.filepath == other.filepath[:len(self.filepath)] def has_both(self): return len(self.filepath) > 0 and len(self.chunkpath) > 0 @property def filepaths(self): for index in range(len(self.filepath)): yield ( self.filepath[index], Path(self.filepath[:index+1], []) ) @property def chunkpaths(self): for index in range(len(self.chunkpath)): yield ( self.chunkpath[index], Path(self.filepath[:], self.chunkpath[:index+1]) )
def rename_path(self, path, name): with self.modify("Rename path") as document_dict: for p in document_dict.paragraph_dict_iterator(): if p["type"] == "code": filelen = len(p["filepath"]) chunklen = len(p["chunkpath"]) if path.is_prefix(Path(p["filepath"], p["chunkpath"])): if path.length > filelen: p["chunkpath"][path.length-1-filelen] = name else: p["filepath"][path.length-1] = name else: for f in p["fragments"]: if f["type"] == "chunk": if path.is_prefix(Path(p["filepath"], p["chunkpath"]+f["path"])): f["path"][path.length-1-filelen-chunklen] = name
@property def fragments(self): return CodeFragment.create_list( self._document, self, self._paragraph_dict["fragments"] ) def iter_code_fragments(self): return iter(self.fragments)
class CodeFragment(object): @staticmethod def create_list(document, code_paragraph, code_fragment_dicts): return [ CodeFragment.create(document, code_paragraph, code_fragment_dict) for code_fragment_dict in code_fragment_dicts ] @staticmethod def create(document, code_paragraph, code_fragment_dict): return { "variable": VariableCodeFragment, "chunk": ChunkCodeFragment, "code": CodeCodeFragment, "tabstop": TabstopCodeFragment, }.get(code_fragment_dict["type"], CodeFragment)(document, code_paragraph, code_fragment_dict) def __init__(self, document, code_paragraph, code_fragment_dict): self._document = document self._code_paragraph = code_paragraph self._code_fragment_dict = code_fragment_dict @property def type(self): return self._code_fragment_dict["type"]
They can be converted to text for editing:
@property def text_version(self): self._variable_map = {} text_version = TextVersion() for fragment in self.fragments: fragment.fill_text_version(text_version) return text_version.text def add_variable_map(self, name, id_): entry = name index = 1 while entry in self._variable_map and self._variable_map[entry] != id_: entry = "{}{}".format(name, index) index += 1 self._variable_map[entry] = id_ return entry
And converted back again from text:
@text_version.setter def text_version(self, value): with self.multi_update(): self.update({ "fragments": self._parse(value) }) def _parse(self, value): self._parsed_fragments = [] self._parse_buffer = "" for line in value.splitlines(): match = re.match(self._chunk_fragment_re(), line) if match: self._parse_clear() body_match = re.match(r"^(.*?)(, blank_lines_before=(\d+))?$", match.group(2)) if body_match.group(2): blank_lines_before = int(body_match.group(3)) else: blank_lines_before = 0 self._parsed_fragments.append({ "type": "chunk", "path": body_match.group(1).split("/"), "prefix": match.group(1), "blank_lines_before": blank_lines_before, }) else: while line: variable_match = re.match(self._variable_fragment_re(), line) tabstop_match = re.match(self._tabstop_fragment_re(), line) if variable_match: self._parse_clear() self._parsed_fragments.append({ "type": "variable", "id": self._get_variable_id(variable_match.group(1)) }) line = line[len(variable_match.group(0)):] elif tabstop_match: self._parse_clear() self._parsed_fragments.append({ "type": "tabstop", "index": int(tabstop_match.group(1)) }) line = line[len(tabstop_match.group(0)):] else: self._parse_buffer += line[0] line = line[1:] self._parse_buffer += "\n" self._parse_clear() return self._parsed_fragments def _get_variable_id(self, identifier): if identifier in self._variable_map: return self._variable_map[identifier] elif self._document.lookup_variable(identifier) is None: return self._document.new_variable(identifier) else: return identifier def _parse_clear(self): if self._parse_buffer: self._parsed_fragments.append({"type": "code", "text": self._parse_buffer}) self._parse_buffer = "" def _chunk_fragment_re(self): start, end = self.chunk_delimiters return r"^(\s*){}(.*){}\s*$".format(re.escape(start), re.escape(end)) def _variable_fragment_re(self): start, end = self.variable_delimiters return r"{}(.*?){}".format(re.escape(start), re.escape(end)) def _tabstop_fragment_re(self): start, end = self.chunk_delimiters return r"{}{{(\d+)}}{}".format(re.escape(start), re.escape(end))
class VariableCodeFragment(CodeFragment): @property def id(self): return self._code_fragment_dict["id"] @property def name(self): name = self._document.lookup_variable(self.id) if name is None: return self.id else: return name def fill_text_version(self, text_version): start, end = self._code_paragraph.variable_delimiters text_version.add(start) text_version.add(self._code_paragraph.add_variable_map(self.name, self.id)) text_version.add(end)
class ChunkCodeFragment(CodeFragment): @property def prefix(self): return self._code_fragment_dict["prefix"] @property def blank_lines_before(self): return self._code_fragment_dict.get("blank_lines_before", 0) @property def path(self): return copy.deepcopy(self._code_fragment_dict["path"]) def fill_text_version(self, text_version, for_view=False): start, end = self._code_paragraph.chunk_delimiters text_version.add(self.prefix) text_version.add(start) text_version.add("/".join(self.path)) if self.blank_lines_before > 0 and not for_view: text_version.add(", blank_lines_before=") text_version.add(str(self.blank_lines_before)) text_version.add(end) text_version.add("\n")
class CodeCodeFragment(CodeFragment): @property def text(self): return self._code_fragment_dict["text"] @property def text_version(self): text_version = TextVersion() self.fill_text_version(text_version) return text_version.text def fill_text_version(self, text_version): text_version.add(self.text)
class TabstopCodeFragment(CodeFragment): @property def index(self): return self._code_fragment_dict["index"] def fill_text_version(self, text_version): start, end = self._code_paragraph.chunk_delimiters text_version.add(start) text_version.add("{") text_version.add(str(self.index)) text_version.add("}") text_version.add(end)
@property def tokens(self): chain = CharChain() for fragment in self.fragments: if fragment.type == "chunk": text_version = TextVersion() fragment.fill_text_version(text_version, for_view=True) chain.append( text_version.text, token_type=TokenType.Comment.Preproc, subpath=self.path.extend_chunk(fragment.path) ) elif fragment.type == "variable": chain.append(fragment.name, variable=fragment.id) elif fragment.type == "code": chain.append(fragment.text) elif fragment.type == "tabstop": chain.mark_tabstop(fragment.index) chain.align_tabstops() chain.colorize(self.pygments_lexer) return chain.to_tokens()
class CharChain(object): def __init__(self): self.head = None self.tail = None self.tabstops = defaultdict(list) def __iter__(self): char = self.head while char is not None: yield char char = char.next def mark_tabstop(self, index): self.tabstops[index].append(self.tail) def align_tabstops(self): align_length = 0 for index in sorted(self.tabstops.keys()): align_length = max([align_length+1]+[ char.count_chars_to_start_of_line() for char in self.tabstops[index] ]) for char in self.tabstops[index]: x = range(align_length - char.count_chars_to_start_of_line()) for _ in x: self._insert_after(char, Char(" ", {})) def append(self, string, **meta): for char in string: self._insert_after(self.tail, Char(char, dict(meta))) def to_tokens(self): return [Token(char.value, **char.meta) for char in self] def colorize(self, pygments_lexer): pygments_text = self._extract_non_colored_text() pygments_tokens = pygments_lexer.get_tokens(pygments_text) self._apply_pygments_tokens(pygments_tokens) def _extract_non_colored_text(self): return "".join( char.value for char in self if char.meta.get("token_type") is None ) def _apply_pygments_tokens(self, pygments_tokens): char = self.head for pygments_token, text in pygments_tokens: for _ in text: while char is not None and char.meta.get("token_type") is not None: char = char.next if char is not None: char.meta["token_type"] = pygments_token char = char.next def _insert_after(self, before, char): if self.tail is None: self.head = char self.tail = char else: char.next = before.next char.previuos = before if before.next is not None: before.next.previuos = char before.next = char if before is self.tail: self.tail = char class Char(object): def __init__(self, value, meta): self.previuos = None self.next = None self.value = value self.meta = meta def count_chars_to_start_of_line(self): count = 0 char = self while char is not None and not char.is_newline(): char = char.previuos count += 1 return count def is_newline(self): return self.value == "\n"
@property def language(self): return "".join(self.pygments_lexer.aliases[:1]) @property def pygments_lexer(self): try: return pygments.lexers.get_lexer_for_filename( self.filename, stripnl=False ) except: return pygments.lexers.TextLexer(stripnl=False)
@property def chunk_delimiters(self): return ("<<", ">>")
@property def variable_delimiters(self): return ( "__RL_", "__" )
class ImageParagraph(Paragraph): @property def fragments(self): return TextFragment.create_list(self._document, self._paragraph_dict["fragments"]) @property def tokens(self): return [x.token for x in self.fragments] @property def image_base64(self): return self._paragraph_dict.get("image_base64", None) @property def text_version(self): return fragments_to_text(self.fragments) @text_version.setter def text_version(self, value): self.update({"fragments": text_to_fragments(value)}) def iter_text_fragments(self): return iter(self.fragments)
class ExpandedCodeParagraph(Paragraph): @property def tokens(self): paragraph, chain = CodeExpander(self._document).expand_id(self.code_id) if paragraph is not None: chain.colorize(paragraph.pygments_lexer) return chain.to_tokens() @property def code_id(self): return self._paragraph_dict.get("code_id")
class TextFragment(object): @staticmethod def create_list(document, text_fragment_dicts): return [ TextFragment.create(document, text_fragment_dict, index) for index, text_fragment_dict in enumerate(text_fragment_dicts) ] @staticmethod def create(document, text_fragment_dict, index): return { "strong": StrongTextFragment, "emphasis": EmphasisTextFragment, "code": CodeTextFragment, "variable": VariableTextFragment, "reference": ReferenceTextFragment, "link": LinkTextFragment, }.get(text_fragment_dict["type"], TextFragment)(document, text_fragment_dict, index) def __init__(self, document, text_fragment_dict, index): self._document = document self._text_fragment_dict = text_fragment_dict self._index = index @property def type(self): return self._text_fragment_dict["type"] @property def text(self): return self._text_fragment_dict["text"] @property def token(self): return Token(self.text, fragment_index=self._index) def fill_text_version(self, text_version): text_version.add_with_index(self.text, self._index)
class StrongTextFragment(TextFragment): @property def token(self): return Token(self.text, token_type=TokenType.RLiterate.Strong, fragment_index=self._index) def fill_text_version(self, text_version): text_version.add("**") text_version.add_with_index(self.text, self._index) text_version.add("**")
class EmphasisTextFragment(TextFragment): @property def token(self): return Token(self.text, token_type=TokenType.RLiterate.Emphasis, fragment_index=self._index) def fill_text_version(self, text_version): text_version.add("*") text_version.add_with_index(self.text, self._index) text_version.add("*")
class CodeTextFragment(TextFragment): @property def token(self): return Token(self.text, token_type=TokenType.RLiterate.Code, fragment_index=self._index) def fill_text_version(self, text_version): text_version.add("`") text_version.add_with_index(self.text, self._index) text_version.add("`")
class VariableTextFragment(TextFragment): @property def id(self): return self._text_fragment_dict["id"] @property def name(self): name = self._document.lookup_variable(self.id) if name is None: return self.id else: return name def fill_text_version(self, text_version): text_version.add("``") text_version.add_with_index(self.id, self._index) text_version.add("``") @property def token(self): return Token(self.name, token_type=TokenType.RLiterate.Variable, fragment_index=self._index)
class ReferenceTextFragment(TextFragment): @property def page_id(self): return self._text_fragment_dict["page_id"] @property def title(self): if self.text: return self.text if self._document.get_page(self.page_id) is not None: return self._document.get_page(self.page_id).title return self.page_id @property def token(self): return Token(self.title, token_type=TokenType.RLiterate.Reference, page_id=self.page_id, fragment_index=self._index) def fill_text_version(self, text_version): text_version.add("[[") text_version.add_with_index(self.page_id, self._index) if self.text: text_version.add(":") text_version.add(self.text) text_version.add("]]")
class LinkTextFragment(TextFragment): @property def url(self): return self._text_fragment_dict["url"] @property def title(self): if self.text: return self.text return self.url @property def token(self): return Token(self.title, token_type=TokenType.RLiterate.Link, url=self.url, fragment_index=self._index) def fill_text_version(self, text_version): text_version.add("[") text_version.add_with_index(self.text, self._index) text_version.add("]") text_version.add("(") text_version.add(self.url) text_version.add(")")
def new_variable(self, name): with self.modify("New variable") as document_dict: variable_id = genid() document_dict["variables"][variable_id] = name return variable_id def rename_variable(self, variable_id, name): with self.modify("Rename variable") as document_dict: document_dict["variables"][variable_id] = name def lookup_variable(self, variable_id): return self.read_only_document_dict["variables"].get(variable_id)
It is common that paragraphs are converted to text for editing. The edited text then needs to be parsed to convert it back to a "rich object". The TextVersion class helps keep track of where items are rendered to the text version.
class TextVersion(object): def __init__(self): self.text = "" self.indicies = {} self._index_prefix = [] def get_selection(self, index): return self.indicies.get(index, (0, 0)) @contextlib.contextmanager def index(self, index): self._index_prefix.append(index) yield self._index_prefix.pop(-1) def add_with_index(self, text, index): start = len(self.text) end = start + len(text) self.indicies[tuple(self._index_prefix + [index])] = (start, end) self.add(text) def add(self, text): self.text += text
The JSON structure of an RLiterate document changes over time. If a document with an old structure is opened, it is automatically converted to the current structure.
def _convert_to_latest(self, document_dict): for fn in [ <<legacy handlers>> ]: document_dict = fn(document_dict) return document_dict
Paragraphs that contain inline text used to store it with special syntax like this:
INLINE_TEXT = "This **is** very *cool*. [[106af6f8665c45e8ab751993a6abc876:page]] is it. Some `code`. [link](http://example.com)."
This should be converted to fragments:
FRAGMENTS = [ {"type": "text", "text": "This "}, {"type": "strong", "text": "is"}, {"type": "text", "text": " very "}, {"type": "emphasis", "text": "cool"}, {"type": "text", "text": ". "}, {"type": "reference", "text": "page", "page_id": "106af6f8665c45e8ab751993a6abc876"}, {"type": "text", "text": " is it. Some "}, {"type": "code", "text": "code"}, {"type": "text", "text": ". "}, {"type": "link", "text": "link", "url": "http://example.com"}, {"type": "text", "text": "."}, ]
This conversion needs to be done for paragraph types text, quote, image, and list. In addition, the list paragraph used to store the whole list in the text attribute. In order to convert list item text to fragments, the list structure also needs to change.
Here is how text paragraphs change:
TEXT_OLD = {"id": "abc1", "type": "text", "text": INLINE_TEXT} TEXT_NEW = {"id": "abc1", "type": "text", "fragments": FRAGMENTS}
Here is how quote paragraphs change:
QUOTE_OLD = {"id": "abc2", "type": "quote", "text": INLINE_TEXT} QUOTE_NEW = {"id": "abc2", "type": "quote", "fragments": FRAGMENTS}
Here is how image paragraphs change:
IMAGE_OLD = {"id": "abc3", "type": "image", "image_base64": "data", "text": INLINE_TEXT} IMAGE_NEW = {"id": "abc3", "type": "image", "image_base64": "data", "fragments": FRAGMENTS}
Here is how list paragraphs change:
LIST_OLD = { "id": "abc4", "type": "list", "text": "\n".join([ "* **a**{}".format(INLINE_TEXT), "* **b**{}".format(INLINE_TEXT), " 1. **c**{}".format(INLINE_TEXT), " 1. **d**{}".format(INLINE_TEXT), ]), } LIST_NEW = { "id": "abc4", "type": "list", "child_type": "unordered", "children": [ { "fragments": [{"type": "strong", "text": "a"}]+FRAGMENTS, "child_type": None, "children": [], }, { "fragments": [{"type": "strong", "text": "b"}]+FRAGMENTS, "child_type": "ordered", "children": [ { "fragments": [{"type": "strong", "text": "c"}]+FRAGMENTS, "child_type": None, "children": [], }, { "fragments": [{"type": "strong", "text": "d"}]+FRAGMENTS, "child_type": None, "children": [], }, ], }, ] }
Here is a test that ensures that the paragraphs are properly converted:
def test_converts_inline_text_to_fragments(tmpfile): old_paragraphs = [TEXT_OLD, QUOTE_OLD, IMAGE_OLD, LIST_OLD] new_paragraphs = [TEXT_NEW, QUOTE_NEW, IMAGE_NEW, LIST_NEW] document = write_read_document(tmpfile, { "title": "Root", "id": "abc1", "paragraphs": old_paragraphs, "children": [ { "title": "Child", "children": [], "id": "abc2", "paragraphs": old_paragraphs, } ], }) assert document["root_page"]["paragraphs"] == new_paragraphs assert document["root_page"]["children"][0]["paragraphs"] == new_paragraphs
Here is the implementation:
self._legacy_inline_text,
def _legacy_inline_text(self, document_dict): for paragraph in document_dict.get("paragraphs", []): if paragraph["type"] in ["text", "quote", "image"] and "text" in paragraph: paragraph["fragments"] = LegacyInlineTextParser().parse(paragraph["text"]) del paragraph["text"] elif paragraph["type"] in ["list"] and "text" in paragraph: paragraph["child_type"], paragraph["children"] = LegacyListParser(paragraph["text"]).parse_items() del paragraph["text"] for child in document_dict.get("children", []): self._legacy_inline_text(child) return document_dict
class LegacyInlineTextParser(object): SPACE_RE = re.compile(r"\s+") PATTERNS = [ ( re.compile(r"\*\*(.+?)\*\*", flags=re.DOTALL), lambda parser, match: { "type": "strong", "text": match.group(1), } ), ( re.compile(r"\*(.+?)\*", flags=re.DOTALL), lambda parser, match: { "type": "emphasis", "text": match.group(1), } ), ( re.compile(r"`(.+?)`", flags=re.DOTALL), lambda parser, match: { "type": "code", "text": match.group(1), } ), ( re.compile(r"\[\[(.+?)(:(.+?))?\]\]", flags=re.DOTALL), lambda parser, match: { "type": "reference", "text": match.group(3), "page_id": match.group(1), } ), ( re.compile(r"\[(.*?)\]\((.+?)\)", flags=re.DOTALL), lambda parser, match: { "type": "link", "text": match.group(1), "url": match.group(2), } ), ] def parse(self, text): text = self._normalise_space(text) fragments = [] partial = "" while text: result = self._get_special_fragment(text) if result is None: partial += text[0] text = text[1:] else: match, fragment = result if partial: fragments.append({"type": "text", "text": partial}) partial = "" fragments.append(fragment) text = text[match.end(0):] if partial: fragments.append({"type": "text", "text": partial}) return fragments def _normalise_space(self, text): return self.SPACE_RE.sub(" ", text).strip() def _get_special_fragment(self, text): for pattern, fn in self.PATTERNS: match = pattern.match(text) if match: return match, fn(self, match)
class LegacyListParser(object): ITEM_START_RE = re.compile(r"( *)([*]|\d+[.]) (.*)") def __init__(self, text): self.lines = text.strip().split("\n") def parse_items(self, level=0): items = [] list_type = None while True: type_and_item = self.parse_item(level) if type_and_item is None: return list_type, items else: item_type, item = type_and_item if list_type is None: list_type = item_type items.append(item) def parse_item(self, level): parts = self.consume_bodies() next_level = level + 1 item_type = None if self.lines: match = self.ITEM_START_RE.match(self.lines[0]) if match: matched_level = len(match.group(1)) if matched_level >= level: parts.append(match.group(3)) self.lines.pop(0) parts.extend(self.consume_bodies()) next_level = matched_level + 1 if "*" in match.group(2): item_type = "unordered" else: item_type = "ordered" if parts: child_type, children = self.parse_items(next_level) return (item_type, { "fragments": LegacyInlineTextParser().parse(" ".join(parts)), "children": children, "child_type": child_type, }) def consume_bodies(self): bodies = [] while self.lines: if self.ITEM_START_RE.match(self.lines[0]): break else: bodies.append(self.lines.pop(0)) return bodies
Code paragraphs used to have path and text attributes like this:
CODE_OLD = { "id": "abc1", "type": "code", "path": ["foo.py", "<<functions>>"], "text": "one\n<<chunk>>\ntwo\n", }
Those should be converted to filepath, chunkpath, and fragments attributes:
CODE_NEW = { "id": "abc1", "type": "code", "filepath": ["foo.py"], "chunkpath": ["functions"], "fragments": [ {"type": "code", "text": "one\n"}, {"type": "chunk", "prefix": "", "path": ["chunk"]}, {"type": "code", "text": "two\n"}, ], }
Here is a test that ensures that the conversion is done in all pages:
def test_converts_legacy_code_paragraphs(tmpfile): document = write_read_document(tmpfile, { "title": "Root", "children": [ { "title": "Child", "children": [], "id": "abc2", "paragraphs": [CODE_OLD], }, ], "id": "abc1", "paragraphs": [CODE_OLD], }) assert document["root_page"]["paragraphs"] == [CODE_NEW] assert document["root_page"]["children"][0]["paragraphs"] == [CODE_NEW]
The implementation:
self._legacy_inline_code,
def _legacy_inline_code(self, document_dict): for p in document_dict.get("paragraphs", []): if p["type"] == "code" and "path" in p: p["filepath"], p["chunkpath"] = split_legacy_path(p["path"]) del p["path"] if p["type"] == "code" and "text" in p: p["fragments"] = legacy_code_text_to_fragments(p["text"]) del p["text"] for child in document_dict.get("children", []): self._legacy_inline_code(child) return document_dict
More details about the path split function:
@pytest.mark.parametrize("path,expected_split", [ ( ["foo.py", "<<functions>>"], (["foo.py"], ["functions"]) ), ( ["foo.py", "<<Foo/__init__>>"], (["foo.py"], ["Foo", "__init__"]) ), ( ["foo.py", "<<functions>>", "<<with", "missing>>", "stuff/or"], (["foo.py"], ["functions", "with", "missing", "stuff", "or"]) ), ( ["foo.py", "<<with", "missing>>", "stuff/or"], (["foo.py", "<<with", "missing>>", "stuff", "or"], []) ), ( ["bar/foo.py"], (["bar", "foo.py"], []) ), ( [], ([], []) ), ]) def test_split_legacy_path(path, expected_split): assert split_legacy_path(path) == expected_split
def split_legacy_path(path): filepath = [] chunkpath = [] while path and not (path[0].startswith("<<") and path[0].endswith(">>")): filepath.extend(path.pop(0).split("/")) for chunk in path: if chunk.startswith("<<"): chunk = chunk[2:] if chunk.endswith(">>"): chunk = chunk[:-2] chunkpath.extend(chunk.split("/")) return filepath, chunkpath
More details about the text to fragments function:
@pytest.mark.parametrize("text,expected_fragments", [ ( "def foo()\n", [ {"type": "code", "text": "def foo()\n"}, ] ), ( "def foo()\n <<foo/pre>>\n return False", [ {"type": "code", "text": "def foo()\n"}, {"type": "chunk", "prefix": " ", "path": ["foo", "pre"]}, {"type": "code", "text": " return False\n"}, ] ), ]) def test_legacy_code_text_to_fragments(text, expected_fragments): assert legacy_code_text_to_fragments(text) == expected_fragments
def legacy_code_text_to_fragments(text): fragments = [] current_text = "" for line in text.splitlines(): match = re.match(r"^(\s*)<<(.*)>>\s*$", line) if match: if current_text: fragments.append({"type": "code", "text": current_text}) current_text = "" fragments.append({"type": "chunk", "path": match.group(2).split("/"), "prefix": match.group(1)}) else: current_text += line current_text += "\n" if current_text: fragments.append({"type": "code", "text": current_text}) return fragments
The root JSON object used to represent the root page. It changed to represent a document so that variables could also be stored somewhere.
def test_converts_legacy_root_document(tmpfile): root_page = { "title": "Root", "children": [], "id": "abc1", "paragraphs": [], } assert write_read_document(tmpfile, root_page) == { "root_page": root_page, "variables": {}, }
The implementation:
self._legacy_root_page,
def _legacy_root_page(self, document_dict): if "root_page" not in document_dict: return { "root_page": document_dict, "variables": {}, } else: return document_dict
The write_read_document function is used in legacy tests to write a legacy document, read it, write it, and then ensure that the written is according to new structure.
def write_read_document(path, document): write_json_to_file(path, document) doc = Document.from_file(path) with doc.notify(): pass return load_json_from_file(path)
Different parts of a docunet requires unique ids. We use uuid and assume that we will never get collisions.
def genid(): return uuid.uuid4().hex
The UI model consists of classes that models the non-specific part of the GUI.
A project is a container for a few other objects:
class Project(Observable): def __init__(self, filepath): Observable.__init__(self) <<Project/__init__>> self.title="{} ({})".format( os.path.basename(filepath), os.path.dirname(os.path.abspath(filepath)) ) self.document = Document.from_file(filepath) self.document.listen(self.notify_forwarder()) self.layout = Layout.from_file(os.path.join( os.path.dirname(filepath), ".{}.layout".format(os.path.basename(filepath)) )) self.layout.listen(self.notify_forwarder()) self.global_settings = GlobalSettings.from_file( os.path.join( wx.StandardPaths.Get().GetUserConfigDir(), ".rliterate.settings" ) ) self.global_settings.listen(self.notify_forwarder()) FileGenerator(self.document) <<Project>>
@property def theme(self): return self.global_settings
@property def selection(self): return Selection()
Wrapper methods for document:
def get_page(self, *args, **kwargs): return self.document.get_page(*args, **kwargs) def iter_pages(self, *args, **kwargs): return self.document.iter_pages(*args, **kwargs) def get_paragraph(self, *args, **kwargs): return self.document.get_paragraph(*args, **kwargs) def add_page(self, *args, **kwargs): return self.document.add_page(*args, **kwargs) def add_paragraph(self, *args, **kwargs): return self.document.add_paragraph(*args, **kwargs) def get_undo_operation(self, *args, **kwargs): return self.document.get_undo_operation(*args, **kwargs) def get_redo_operation(self, *args, **kwargs): return self.document.get_redo_operation(*args, **kwargs) def rename_path(self, *args, **kwargs): return self.document.rename_path(*args, **kwargs) def lookup_variable(self, *args, **kwargs): return self.document.lookup_variable(*args, **kwargs) def rename_variable(self, *args, **kwargs): return self.document.rename_variable(*args, **kwargs)
Wrapper methods for layout:
def toggle_collapsed(self, *args, **kwargs): return self.layout.toggle_collapsed(*args, **kwargs) def is_collapsed(self, *args, **kwargs): return self.layout.is_collapsed(*args, **kwargs) @property def columns(self): return self.layout.columns def is_open(self, *args, **kwargs): return self.layout.is_open(*args, **kwargs) def open_pages(self, *args, **kwargs): if self.active_editor is None: return self.layout.open_pages(*args, **kwargs) else: raise EditInProgress() def can_back(self, *args, **kwargs): return self.layout.can_back(*args, **kwargs) def back(self, *args, **kwargs): return self.layout.back(*args, **kwargs) def can_forward(self, *args, **kwargs): return self.layout.can_forward(*args, **kwargs) def forward(self, *args, **kwargs): return self.layout.forward(*args, **kwargs) @property def hoisted_page(self): return self.layout.hoisted_page @hoisted_page.setter def hoisted_page(self, value): self.layout.hoisted_page = value
Wrapper for theme:
def get_style(self, *args, **kwargs): return SolarizedTheme().get_style(*args, **kwargs)
Misc:
self._active_editor = None
@property def active_editor(self): return self._active_editor @active_editor.setter def active_editor(self, editor): with self.notify(): self._active_editor = editor
self._highlighted_variable = None
@property def highlighted_variable(self): return self._highlighted_variable @highlighted_variable.setter def highlighted_variable(self, variable_id): with self.notify(): self._highlighted_variable = variable_id
class EditInProgress(Exception): pass
class Selection(object): def __init__(self, value=None, trail=None): self.value = value self.trail = [] if trail is None else trail @property def present(self): return self.value is not None def get(self, key): return Selection( self.value.get(key) if self.value is not None else None, trail=[key]+self.trail ) def create(self, value): for key in self.trail: value = {key: value} return Selection(value)
class GlobalSettings(JsonSettings): page_body_width = JsonSettings.property( "theme.page_body_width", 600 ) page_padding = JsonSettings.property( "theme.page_padding", 15 ) shadow_size = JsonSettings.property( "theme.shadow_size", 2 ) paragraph_space = JsonSettings.property( "theme.paragraph_space", 15 ) container_border = JsonSettings.property( "theme.container_border", 15 ) editor_font = JsonSettings.property( "theme.fonts.editor", {"monospace": True, "size": 9} ) toc_font = JsonSettings.property( "theme.fonts.toc", {"size": 10} ) title_font = JsonSettings.property( "theme.fonts.title", {"size": 16} ) code_font = JsonSettings.property( "theme.fonts.code", {"monospace": True} ) text_font = JsonSettings.property( "theme.fonts.text", {"size": 10} ) workspace_background = JsonSettings.property( "theme.workspace.background", "#cccccc" )
A layout knows the visual state of the program. It for example knows what pages are expanded/collapsed in the table of contents and what is shown in the workspace.
class Layout(JsonSettings): def __init__(self, *args, **kwargs): JsonSettings.__init__(self, *args, **kwargs) <<Layout/handle legacy settings dict>> <<Layout/__init__>> <<Layout>>
The hoisted page is stored in toc.hoisted_page_id:
hoisted_page = JsonSettings.property( "toc.hoisted_page_id", None )
The collapsed pages are stored in toc.collapsed:
def is_collapsed(self, page_id): return page_id in self.get("toc.collapsed", []) def toggle_collapsed(self, page_id): collapsed = self.get("toc.collapsed", []) if page_id in collapsed: collapsed.remove(page_id) else: collapsed.append(page_id) self.set("toc.collapsed", collapsed)
The pages displayed in the columns in the workspace are stored in workspace.columns:
self._workspace_columns_history = History( self.columns, size=20 )
columns = JsonSettings.property( "workspace.columns", [] ) def open_pages(self, page_ids, column_index=None): with self._workspace_columns_history.new_value() as value: if column_index is None: column_index = len(self.columns) value[column_index:] = [page_ids[:]] self.columns = value def can_back(self): return self._workspace_columns_history.can_back() def back(self): self._workspace_columns_history.back() self.columns = self._workspace_columns_history.value def can_forward(self): return self._workspace_columns_history.can_forward() def forward(self): self._workspace_columns_history.forward() self.columns = self._workspace_columns_history.value def is_open(self, page_id): for column in self.columns: if page_id in column: return True return False
Scratch pages where replaced by columns.
def test_reads_legacy_scratch_pages(tmpfile): write_json_to_file(tmpfile, { "workspace": { "scratch": ["abc"], } }) layout = Layout.from_file(tmpfile) with layout.notify(): pass assert load_json_from_file(tmpfile)["workspace"] == { "columns": [ ["abc"], ] }
if "workspace" in self._settings_dict: workspace = self._settings_dict["workspace"] if "scratch" in workspace: if "columns" not in workspace: workspace["columns"] = [workspace["scratch"]] del workspace["scratch"]
Some parts of the application can be themed. Token types from pygments denote different things that can be styled.
class BaseTheme(object): def get_style(self, token_type): if token_type in self.styles: return self.styles[token_type] return self.get_style(token_type.parent)
class Style(object): def __init__(self, color, bold=None, underlined=None, italic=False, monospace=False): self.color = color self.color_rgb = tuple([ int(x, 16) for x in (color[1:3], color[3:5], color[5:7]) ]) self.bold = bold self.underlined = underlined self.italic = italic self.monospace = monospace def apply_to_wx_dc(self, dc, base_font, highlight=False): font = base_font if self.bold: font = font.Bold() if self.underlined: font = font.Underlined() if self.italic: font = font.Italic() if self.monospace: font = wx.Font( pointSize=font.GetPointSize(), family=wx.FONTFAMILY_TELETYPE, style=font.GetStyle(), weight=font.GetWeight(), underline=font.GetUnderlined(), ) if highlight: dc.SetTextForeground("#fcf4df") dc.SetTextBackground("#b58900") dc.SetBackgroundMode(wx.SOLID) else: dc.SetTextForeground(self.color_rgb) dc.SetBackgroundMode(wx.TRANSPARENT) dc.SetFont(font)
Here is a theme based on solarized. Mostly stolen from https://github.com/honza/solarized-pygments/blob/master/solarized.py.
class SolarizedTheme(BaseTheme): base03 = "#002b36" base02 = "#073642" base01 = "#586e75" base00 = "#657b83" base0 = "#839496" base1 = "#93a1a1" base2 = "#eee8d5" base3 = "#fdf6e3" yellow = "#b58900" orange = "#cb4b16" red = "#dc322f" magenta = "#d33682" violet = "#6c71c4" blue = "#268bd2" cyan = "#2aa198" green = "#859900" text = "#2e3436" styles = { TokenType: Style(color=base00), TokenType.Keyword: Style(color=green), TokenType.Keyword.Constant: Style(color=cyan), TokenType.Keyword.Declaration: Style(color=blue), TokenType.Keyword.Namespace: Style(color=orange), TokenType.Name.Builtin: Style(color=red), TokenType.Name.Builtin.Pseudo: Style(color=blue), TokenType.Name.Class: Style(color=blue), TokenType.Name.Decorator: Style(color=blue), TokenType.Name.Entity: Style(color=violet), TokenType.Name.Exception: Style(color=yellow), TokenType.Name.Function: Style(color=blue), TokenType.String: Style(color=cyan), TokenType.Number: Style(color=cyan), TokenType.Operator.Word: Style(color=green), TokenType.Comment: Style(color=base1), TokenType.Comment.Preproc: Style(color=magenta, bold=True), TokenType.RLiterate: Style(color=text), TokenType.RLiterate.Emphasis: Style(color=text, italic=True), TokenType.RLiterate.Strong: Style(color=text, bold=True), TokenType.RLiterate.Code: Style(color=text, monospace=True), TokenType.RLiterate.Variable: Style(color=text, monospace=True, italic=True), TokenType.RLiterate.Link: Style(color=blue, underlined=True), TokenType.RLiterate.Reference: Style(color=blue, italic=True), TokenType.RLiterate.Path: Style(color=text, italic=True, bold=True), TokenType.RLiterate.Chunk: Style(color=magenta, bold=True), TokenType.RLiterate.Sep: Style(color=base1), }
The main GUI is written in wxPython. I chose it for two reasons:
The main frame lays out two widgets horizontally: the table of contents (Table of contents) and the workspace (Workspace). It also creates the project from the specified file path.
We put all components in a panel, and let the panel be the only child of the top level window. Otherwise tab cycling will not work properly. Reference?
class MainFrame(wx.Frame, BoxSizerMixin): def __init__(self, project): wx.Frame.__init__( self, None, size=(920, 500), title="{} - RLiterate".format(project.title) ) BoxSizerMixin.__init__(self, wx.HORIZONTAL) self.project = project self.AppendChild(self._create_main_panel(), flag=wx.EXPAND, proportion=1) def _create_main_panel(self): panel = HorizontalPanel(self) self.SetToolBar(ToolBar(panel, self.project)) self._focus_panel = panel.AppendChild( wx.Panel(self) ) panel.AppendChild( TableOfContents(panel, self.project), flag=wx.EXPAND, proportion=0 ) panel.AppendChild( Workspace(panel, self.project), flag=wx.EXPAND, proportion=1 ) return panel <<MainFrame>>
When a child control re-renders itself, the current focus can be lost. If we don't re-focus something, no window will have focus and keyboard shortcuts will not work. (This one took me a few hours to figure out.) Setting focus to items in a scrolling container will cause automatic scroll when focused. We therefore use a dummy focus panel of zero size that gains focus. But if an editor is shown, it has already been given focus, and we don't need to do anything.
def ChildReRendered(self): self.Layout() if self.project.active_editor is None: self._focus_panel.SetFocus()
The ToolBar has two responsiblities:
class ToolBar(wx.ToolBar): def __init__(self, parent, project, *args, **kwargs): wx.ToolBar.__init__(self, parent, *args, **kwargs) self._init_project(project) self._init_tools() def _init_project(self, project): self.project = project self.project.listen(lambda: self._tool_groups.populate(self) ) def _init_tools(self): main_frame = self.GetTopLevelParent() self._tool_groups = ToolGroups(main_frame) <<ToolBar/init tools>> self._tool_groups.populate(self)
ToolGroups maintains a list of groups of tools that can be populated into a tool bar:
class ToolGroups(object): def __init__(self, frame): self._frame = frame self._tool_groups = [] def add_group(self, *args, **kwargs): group = ToolGroup(self._frame, *args, **kwargs) self._tool_groups.append(group) return group def populate(self, toolbar): items = [] toolbar.ClearTools() first = True for group in self._tool_groups: if group.is_active(): if not first: toolbar.AddSeparator() first = False group.populate(toolbar) items.extend(group.accelerator_entries()) toolbar.Realize() self._frame.SetAcceleratorTable(wx.AcceleratorTable(items))
ToolGroup maintains a list of tools that can be shown or not in the toolbar depending on the active_fn:
class ToolGroup(object): def __init__(self, frame, active_fn=None): self._tools = [] self._frame = frame self._active_fn = active_fn def add_tool(self, *args, **kwargs): self._tools.append(Tool(self._frame, *args, **kwargs)) def is_active(self): if self._active_fn is None: return True else: return self._active_fn() def accelerator_entries(self): entries = [] for tool in self._tools: entries.extend(tool.accelerator_entries()) return entries def populate(self, toolbar): for tool in self._tools: tool.populate(toolbar)
Tool has information about a single tool and knows how to populate itselt in a tool bar:
class Tool(object): def __init__(self, frame, art, action_fn, short_help="", enabled_fn=None, shortcuts=[]): self.id = wx.NewId() frame.Bind(wx.EVT_MENU, self._on_action, id=self.id) self.art = art self.short_help = short_help self.action_fn = action_fn self.enabled_fn = enabled_fn self.shortcuts = shortcuts def _on_action(self, event): if self.action_fn is not None: self.action_fn() def _is_enabled(self): if self.enabled_fn is None: return True else: return self.enabled_fn() def accelerator_entries(self): return [wx.AcceleratorEntry(a, b, self.id) for (a, b) in self.shortcuts] def populate(self, toolbar): toolbar.AddSimpleTool( self.id, wx.ArtProvider.GetBitmap( self.art, wx.ART_BUTTON, (24, 24) ) ) toolbar.EnableTool(self.id, self._is_enabled()) if callable(self.short_help): text = self.short_help() else: text = self.short_help if self.shortcuts: text = "{} ({})".format( text, " / ".join(x.ToString() for x in self.accelerator_entries()) ) toolbar.SetToolShortHelp(self.id, text)
editor_group = self._tool_groups.add_group( lambda: self.project.active_editor is not None )
Save:
editor_group.add_tool( wx.ART_FILE_SAVE, lambda: self.project.active_editor.Save(), short_help="Save", shortcuts=[ (wx.ACCEL_CTRL, ord('S')), (wx.ACCEL_CTRL, wx.WXK_RETURN), ] )
Cancel:
editor_group.add_tool( wx.ART_CROSS_MARK, lambda: self.project.active_editor.Cancel(), short_help="Cancel", shortcuts=[ (wx.ACCEL_CTRL, ord('G')), (wx.ACCEL_NORMAL, wx.WXK_ESCAPE), ] )
navigation_group = self._tool_groups.add_group( lambda: self.project.active_editor is None )
Back:
navigation_group.add_tool( wx.ART_GO_BACK, lambda: self.project.back(), short_help="Go back", enabled_fn=lambda: self.project.can_back() )
Forward:
navigation_group.add_tool( wx.ART_GO_FORWARD, lambda: self.project.forward(), short_help="Go forward", enabled_fn=lambda: self.project.can_forward() )
undo_group = self._tool_groups.add_group( lambda: self.project.active_editor is None )
Undo:
undo_group.add_tool( wx.ART_UNDO, lambda: self.project.get_undo_operation()[1](), short_help=lambda: "Undo" if self.project.get_undo_operation() is None else "Undo '{}'".format(self.project.get_undo_operation()[0]), enabled_fn=lambda: self.project.get_undo_operation() is not None, shortcuts=[ (wx.ACCEL_CTRL, ord('Z')), ] )
Redo:
undo_group.add_tool( wx.ART_REDO, lambda: self.project.get_redo_operation()[1](), short_help=lambda: "Redo" if self.project.get_redo_operation() is None else "Redo '{}'".format(self.project.get_redo_operation()[0]), enabled_fn=lambda: self.project.get_redo_operation() is not None )
quit_group = self._tool_groups.add_group()
quit_group.add_tool( wx.ART_QUIT, lambda: main_frame.Close(), short_help="Quit", shortcuts=[ (wx.ACCEL_CTRL, ord('Q')), ] )
As soon as something changes in the project, many widgets are re-rendered. Instead of passing information of what changed, the widgets are just notified that something changed. It is up to the widgets to "figure out" what has changed and find an optimal way to redraw itself.
The _render method sets up things one time. The _re_render method then figures out the optimal way to change itself.
The table of contents shows the outline of the document. It allows only subtrees to be shown (hoisting) and allows subtrees to be expanded/collapsed. It also provides navigation functions to allow pages to be opened.
The main table of contents widget listens for changes to a project (only events related to changes in the document and the layout of the table of contents) and then re-renders itself.
class TableOfContents(VerticalPanel): def __init__(self, parent, project): VerticalPanel.__init__(self, parent, size=(250, -1)) self._init_project(project) <<TableOfContents/__init__>> self._render() def _init_project(self, project): self.project = project self.project.listen(self._re_render) <<TableOfContents>>
The table of contents widget lays out two components in a vertical container: the unhoist button and the page container.
The scroll position of the page container is not preserved if it is destroyed and re-created. We therefore maintain the same instance across re-renders.
def _render(self): with flicker_free_drawing(self): self.unhoist_button = None self.page_container = self.AppendChild( VerticalScrolledWindow(self), flag=wx.EXPAND, proportion=1 ) self.SetBackgroundColour((255, 255, 255)) self._re_render() def _re_render(self): with flicker_free_drawing(self): self.drop_points = [] if self.unhoist_button is not None: self.unhoist_button.Destroy() self.unhoist_button = None self.page_container.RemoveChildren() self._render_unhoist_button() self._render_page_container() self.GetTopLevelParent().ChildReRendered()
The unhoist button (only shown if a page has been hoisted):
def _render_unhoist_button(self): if self._get_hoisted_page() is not None: self.unhoist_button = self.InsertChild( 0, wx.Button(self, label="unhoist"), flag=wx.EXPAND ) self.unhoist_button.Bind( wx.EVT_BUTTON, lambda event: setattr(self.project, "hoisted_page", None) )
The page container is a scrolling container that contains a set of rows representing pages. Each row is appropriately intendent to create the illusion of a tree.
def _render_page_container(self): if self._get_hoisted_page() is None: self._render_page(self.project.get_page()) else: self._render_page(self._get_hoisted_page()) def _get_hoisted_page(self): if self.project.hoisted_page is None: return None else: return self.project.get_page(self.project.hoisted_page) def _render_page(self, page, indentation=0): is_collapsed = self.project.is_collapsed(page.id) self.page_container.AppendChild( TableOfContentsRow(self.page_container, self.project, page, indentation), flag=wx.EXPAND ) divider = self.page_container.AppendChild( Divider(self.page_container, padding=0, height=2), flag=wx.EXPAND ) if is_collapsed or len(page.children) == 0: before_page_id = None else: before_page_id = page.children[0].id self.drop_points.append(TableOfContentsDropPoint( divider=divider, indentation=indentation+1, parent_page_id=page.id, before_page_id=before_page_id )) if not is_collapsed: for child, next_child in pairs(page.children): divider = self._render_page(child, indentation+1) self.drop_points.append(TableOfContentsDropPoint( divider=divider, indentation=indentation+1, parent_page_id=page.id, before_page_id=None if next_child is None else next_child.id )) return divider
class TableOfContentsDropPoint(object): def __init__(self, divider, indentation, parent_page_id, before_page_id): self.divider = divider self.indentation = indentation self.parent_page_id = parent_page_id self.before_page_id = before_page_id def x_distance_to(self, x): left_padding = TableOfContentsButton.SIZE+1+TableOfContentsRow.BORDER span_x_center = left_padding + TableOfContentsRow.INDENTATION_SIZE * (self.indentation + 1.5) return abs(span_x_center - x) def y_distance_to(self, y): return abs(self.divider.Position.y + self.divider.Size[1]/2 - y) def Show(self): self.divider.Show(sum([ TableOfContentsRow.BORDER, TableOfContentsButton.SIZE, 1, self.indentation*TableOfContentsRow.INDENTATION_SIZE, ])) def Hide(self): self.divider.Hide()
def pairs(items): return zip(items, items[1:]+[None])
Inside the table of contents, pages can be dragged and drop. The drag is initiated in the row widget and handled in the table of contents widget.
self.SetDropTarget(TableOfContentsDropTarget(self, self.project))
class TableOfContentsDropTarget(DropPointDropTarget): def __init__(self, toc, project): DropPointDropTarget.__init__(self, toc, "page") self.project = project def OnDataDropped(self, dropped_page, drop_point): self.project.get_page(dropped_page["page_id"]).move( parent_page_id=drop_point.parent_page_id, before_page_id=drop_point.before_page_id )
The DropPointDropTarget (Drop point drop target) requires FindClosestDropPoint to be defined on the target object. Here it is:
def FindClosestDropPoint(self, screen_pos): client_pos = (client_x, client_y) = self.page_container.ScreenToClient(screen_pos) if self.page_container.HitTest(client_pos) == wx.HT_WINDOW_INSIDE: y_distances = defaultdict(list) for drop_point in self.drop_points: y_distances[drop_point.y_distance_to(client_y)].append(drop_point) if y_distances: return min( y_distances[min(y_distances.keys())], key=lambda drop_point: drop_point.x_distance_to(client_x) )
The row widget renders the page title at the appropriate indentation. If the page has children, an expand/collapse widget is also rendered to the left of the title.
class TableOfContentsRow(HorizontalPanel): def __init__(self, parent, project, page, indentation): HorizontalPanel.__init__(self, parent) self.project = project self.page = page self.indentation = indentation self._render() <<TableOfContentsRow>>
Rendering lays out the expand/collapse button (if the page has children) and the page title in a horizontal sizer. If the page is open, we highlight it with bold font. (For some reason, the font has to be set on the panel and not on the static text in order for size calculations to behave correctly.)
BORDER = 2 INDENTATION_SIZE = 16 def _render(self): self.AppendSpace(self.indentation*self.INDENTATION_SIZE) if self.page.children: self.AppendChild(TableOfContentsButton(self, self.project, self.page), flag=wx.EXPAND|wx.LEFT, border=self.BORDER) else: self.AppendSpace(TableOfContentsButton.SIZE+1+self.BORDER) if self.project.is_open(self.page.id): self.Font = create_font(**dict(self.project.theme.toc_font, bold=True)) else: self.Font = create_font(**self.project.theme.toc_font) text = self.AppendChild(self._label(), flag=wx.ALL, border=self.BORDER) self.Bind(wx.EVT_ENTER_WINDOW, self._on_enter_window) self.Bind(wx.EVT_LEAVE_WINDOW, self._on_leave_window) for helper in [MouseEventHelper(self), MouseEventHelper(text)]: helper.OnClick = self._on_click helper.OnRightClick = self._on_right_click helper.OnDrag = self._on_drag def _label(self): label = wx.StaticText(self) label.SetLabelText(self.page.title) return label
SetLabelText is needed, otherwise '&' characters are not rendered in wx.StaticText.
Event handlers:
def _on_click(self): open_pages_gui(self, self.project, [self.page.id], column_index=0) def _on_right_click(self, event): menu = PageContextMenu(self.project, self.page) self.PopupMenu(menu) menu.Destroy() def _on_drag(self): data = RliterateDataObject("page", { "page_id": self.page.id, }) drag_source = wx.DropSource(self) drag_source.SetData(data) result = drag_source.DoDragDrop(wx.Drag_DefaultMove) def _on_enter_window(self, event): self.SetBackgroundColour((240, 240, 240)) def _on_leave_window(self, event): self.SetBackgroundColour((255, 255, 255))
The expand/collapse widget draws a native looking expand/collapse symbol. When clicked, the corresponding page it toggled.
class TableOfContentsButton(wx.Panel): SIZE = 16 def __init__(self, parent, project, page): wx.Panel.__init__(self, parent, size=(self.SIZE+1, -1)) self.Bind(wx.EVT_PAINT, self.OnPaint) self.Bind(wx.EVT_LEFT_DOWN, self.OnLeftDown) self.project = project self.page = page self.SetCursor(wx.StockCursor(wx.CURSOR_HAND)) def OnLeftDown(self, event): self.project.toggle_collapsed(self.page.id) def OnPaint(self, event): dc = wx.GCDC(wx.PaintDC(self)) dc.SetBrush(wx.BLACK_BRUSH) render = wx.RendererNative.Get() (w, h) = self.Size render.DrawTreeItemButton( self, dc, (0, (h-self.SIZE)/2, self.SIZE, self.SIZE), flags=0 if self.project.is_collapsed(self.page.id) else wx.CONTROL_EXPANDED )
class PageContextMenu(wx.Menu): def __init__(self, project, page): wx.Menu.__init__(self) self.project = project self.page = page self.child_ids = [page.id]+[child.id for child in page.children] self._create_menu() def _create_menu(self): self.Bind( wx.EVT_MENU, lambda event: open_pages_gui(self, self.project, [self.page.id], column_index=0), self.Append(wx.NewId(), "Open") ) self.Bind( wx.EVT_MENU, lambda event: open_pages_gui(self, self.project, [self.page.id]), self.Append(wx.NewId(), "Open append") ) self.Bind( wx.EVT_MENU, lambda event: open_pages_gui(self, self.project, self.child_ids, column_index=0), self.Append(wx.NewId(), "Open with children") ) self.Bind( wx.EVT_MENU, lambda event: open_pages_gui(self, self.project, self.child_ids), self.Append(wx.NewId(), "Open with children append") ) self.AppendSeparator() self.Bind( wx.EVT_MENU, lambda event: self.project.add_page(parent_id=self.page.id), self.Append(wx.NewId(), "Add child") ) self.AppendSeparator() self.Bind( wx.EVT_MENU, lambda event: setattr(self.project, "hoisted_page", self.page.id), self.Append(wx.NewId(), "Hoist") ) self.AppendSeparator() self.Bind( wx.EVT_MENU, lambda event: set_clipboard_text(self.page.id), self.Append(wx.NewId(), "Copy id") ) self.AppendSeparator() delete_item = self.Append(wx.NewId(), "Delete") delete_item.Enable(self.page.id != self.project.get_page().id) self.Bind( wx.EVT_MENU, lambda event: self.page.delete(), delete_item )
def set_clipboard_text(text): if wx.TheClipboard.Open(): try: wx.TheClipboard.SetData(wx.TextDataObject(text.encode("utf-8"))) finally: wx.TheClipboard.Close()
The workspace shows pages in the document. It is organized in columns, where each column can show multiple pages. The pages can also be edited from the workspace.
The main workspace widget renders column widgets horizontally. It can be scrolled left and right if more columns are shown than can fit on the screen. It re-renderes when the project changes.
class Workspace(HorizontalScrolledWindow): def __init__(self, parent, project): HorizontalScrolledWindow.__init__(self, parent, style=wx.HSCROLL) self._init_project(project) <<Workspace/__init__>> self._render() def _init_project(self, project): self.project = project self.project.listen(self._re_render) <<Workspace>>
Rendering a workspace means laying out a set of column widgets horizontally. Columns are filled with page containers. It uses Rendering strategy.
def _render(self): with flicker_free_drawing(self): self.space = self.AppendSpace() self.columns = [] self._re_render()
def _re_render(self): if self.project.active_editor is not None: return with flicker_free_drawing(self): self.SetBackgroundColour(self.project.theme.workspace_background) self._update_space() self._update_columns() self.GetTopLevelParent().ChildReRendered()
def _update_space(self): self.space.SetSize(self.project.theme.page_padding)
def _update_columns(self): column_count_changed = self._ensure_num_columns(len(self.project.columns)) last_column_changed_pages = False for index, page_ids in enumerate(self.project.columns): last_column_changed_pages = self.columns[index].SetPages( page_ids, self.project.selection.get(index) ) if column_count_changed or last_column_changed_pages: wx.CallAfter(self.ScrollToEnd) def _ensure_num_columns(self, num): count_changed = False while len(self.columns) > num: count_changed = True self.columns.pop(-1).Destroy() while len(self.columns) < num: count_changed = True self.columns.append(self._add_column()) return count_changed def _add_column(self): return self.AppendChild( Column(self, project=self.project, index=len(self.columns)), flag=wx.EXPAND )
CallAfter is needed becasue the layout is not "ready" at this point, and sizer reports size (0, 0), so ScrollToEnd does not work.
On removing widgets Removing a widget from its wxPython parent:
Widgets will remove themselves from the sizer when they are destroyed, so yes, it is redundant. Also, widgets will remove themselves from the parent's child list, so callingRemoveChildis also redundant. IOW, the call towindow.Destroy()will be sufficient to remove the widget and its resources, and to do the necessary clean-ups in the parent and the sizer.
Inside a workspace, paragraphs can be dragged and dropped. The drag is handled in the paragraph widget, but the drop is handled in the workspace widget.
self.SetDropTarget(WorkspaceDropTarget(self, self.project))
class WorkspaceDropTarget(DropPointDropTarget): def __init__(self, workspace, project): DropPointDropTarget.__init__(self, workspace, "paragraph") self.project = project def OnDataDropped(self, dropped_paragraph, drop_point): self.project.get_paragraph( dropped_paragraph["page_id"], dropped_paragraph["paragraph_id"] ).move( target_page=drop_point.page_id, before_paragraph=drop_point.next_paragraph_id )
The DropPointDropTarget requires FindClosestDropPoint to be defined on the target object. Here it is:
def FindClosestDropPoint(self, screen_pos): return find_first( self.columns, lambda column: column.FindClosestDropPoint(screen_pos) )
The column widget renders page container widgets vertically. It can be scrolled up and down if more pages are shown than can fit on the screen. It re-renders when SetPages is called.
class Column(VerticalScrolledWindow): def __init__(self, parent, project, index): VerticalScrolledWindow.__init__( self, parent, style=wx.VSCROLL ) self.project = project self.index = index self._render() self.Bind(EVT_HOVERED_TOKEN_CHANGED, self._on_hovered_token_changed) self.Bind(EVT_TOKEN_CLICK, self._on_token_click) <<Column>>
def SetPages(self, page_ids, selection): ids_changed = page_ids != self._page_ids self._page_ids = page_ids self._selection = selection self._re_render() return ids_changed # TODO: return if actual number of rows/containers changed
def _render(self): self._page_ids = [] self._containers = [] self._space = self.AppendSpace() self._re_render()
def _re_render(self): self._adjust_space() self._adjust_size() self._adjust_containers() def _adjust_space(self): self._space.SetSize(self.project.theme.page_padding) def _adjust_size(self): self.MinSize = ( self.project.theme.page_body_width+ 2*self.project.theme.container_border+ self.project.theme.page_padding+ self.project.theme.shadow_size, -1 ) def _adjust_containers(self): num_added = 0 for index, page_id in enumerate(self._page_ids): if self.project.get_page(page_id) is not None: if index >= len(self._containers): self._containers.append( self.AppendChild( PageContainer(self, self.project, page_id, self._selection.get(index)), flag=wx.RIGHT|wx.BOTTOM|wx.EXPAND, border=self.project.theme.page_padding ) ) else: self._containers[index].SetPageId(page_id, self._selection.get(index)) num_added += 1 while len(self._containers) > num_added: self._containers.pop(-1).Destroy() # When? #self.ScrollToBeginning()
def _on_hovered_token_changed(self, event): if event.token is not None and event.token.token_type in [ TokenType.RLiterate.Link, TokenType.RLiterate.Reference, ]: event.widget.SetCursor(wx.StockCursor(wx.CURSOR_HAND)) else: event.widget.SetDefaultCursor() def _on_token_click(self, event): if event.token.token_type == TokenType.RLiterate.Reference: self.OpenPage(event.token.extra["page_id"]) elif event.token.token_type == TokenType.RLiterate.Link: webbrowser.open(event.token.extra["url"]) def OpenPage(self, page_id): open_pages_gui(self, self.project, [page_id], column_index=self.index+1 )
def FindClosestDropPoint(self, screen_pos): return find_first( self._containers, lambda container: container.FindClosestDropPoint(screen_pos) )
The page container widget renders a page frame with a border, and inside the frame it renders a page widget vertically. It re-renders when SetPageId is called.
class PageContainer(VerticalPanel): def __init__(self, parent, project, page_id, selection): VerticalPanel.__init__(self, parent) self.project = project self._render(page_id, selection) <<PageContainer>>
def SetPageId(self, page_id, selection): changed_id = self.page_id != page_id self.page_id = page_id self.selection = selection self._re_render() return changed_id
def _render(self, initial_page_id, selection): self.page_id = initial_page_id self.selection = selection self._render_first_row() self._render_second_row() MouseEventHelper.bind( [ self.inner_container, self.right, self.right_border, self.bottom, self.bottom_border, ], right_click=lambda event: SimpleContextMenu.ShowRecursive(self) ) self._re_render() def _render_first_row(self): first_row = self.AppendChild( HorizontalPanel(self), flag=wx.EXPAND, proportion=1 ) self.inner_container = first_row.AppendChild( VerticalPanel(first_row), flag=wx.EXPAND, proportion=1 ) self.page, self.page_sizer = self.inner_container.AppendChildWithSizer( PagePanel( self.inner_container, self.project, self.page_id, self.selection.get(self.page_id) ), flag=wx.ALL|wx.EXPAND, proportion=1 ) self.right = first_row.AppendChild( VerticalPanel(first_row), flag=wx.EXPAND ) self.right_space = self.right.AppendSpace() self.right_border = self.right.AppendChild( wx.Panel(self.right), flag=wx.EXPAND, proportion=1 ) def _render_second_row(self): self.bottom = self.AppendChild( HorizontalPanel(self), flag=wx.EXPAND ) self.bottom_space = self.bottom.AppendSpace() self.bottom_border = self.bottom.AppendChild( wx.Panel(self.bottom), flag=wx.EXPAND, proportion=1 )
def _re_render(self): self.page.SetPageId(self.page_id, self.selection.get(self.page_id)) self.inner_container.SetBackgroundColour((255, 255, 255)) self.right_border.SetBackgroundColour((150, 150, 150)) self.bottom_border.SetBackgroundColour((150, 150, 150)) self.right.MinSize = (self.project.theme.shadow_size, -1) self.right_space.SetSize(self.project.theme.shadow_size) self.bottom.MinSize = (-1, self.project.theme.shadow_size) self.bottom_space.SetSize(self.project.theme.shadow_size) self.page_sizer.Border = self.project.theme.container_border
def CreateContextMenu(self): menu = SimpleContextMenu("Page") menu.AppendItem("Change width", lambda: SettingsDialog( wx.GetTopLevelParent(self), self.project ).Show() ) return menu
def FindClosestDropPoint(self, screen_pos): return self.page.FindClosestDropPoint(screen_pos)
The page widget renders a title widget, paragraph widgets, and an add button widget vertically. It re-renders when SetPageId is called.
class PagePanel(VerticalPanel): def __init__(self, parent, project, page_id, selection): VerticalPanel.__init__(self, parent) self.project = project self._render(page_id, selection) <<PagePanel>>
def SetPageId(self, page_id, selection): self.RemoveChildren() self._render(page_id, selection)
A page has the following widgets laid out vertically:
def _render(self, initial_page_id, selection): self.page_id = initial_page_id self.selection = selection self.drop_points = [] self.MinSize = ( self.project.theme.page_body_width, -1 ) page = self.project.get_page(self.page_id) divider = self._render_paragraph(Title( self, self.project, page, self.selection.get("title") )) for paragraph in page.paragraphs: self.drop_points.append(PageDropPoint( divider=divider, page_id=self.page_id, next_paragraph_id=paragraph.id )) divider = self._render_paragraph({ "text": Text, "quote": Quote, "list": List, "code": Code, "image": Image, "expanded_code": ExpandedCode, "factory": Factory, }[paragraph.type]( self, self.project, self.page_id, paragraph, self.selection.get("paragraph").get(paragraph.id) )) self.drop_points.append(PageDropPoint( divider=divider, page_id=self.page_id, next_paragraph_id=None )) self._render_add_button() MouseEventHelper.bind([self], right_click=lambda event: SimpleContextMenu.ShowRecursive(self) ) def _render_paragraph(self, paragraph): self.AppendChild(paragraph, flag=wx.EXPAND) return self.AppendChild( Divider( self, padding=(self.project.theme.paragraph_space-3)/2, height=3 ), flag=wx.EXPAND ) def _render_add_button(self): add_button = self.AppendChild( wx.BitmapButton( self, bitmap=wx.ArtProvider.GetBitmap( wx.ART_ADD_BOOKMARK, wx.ART_BUTTON, (16, 16) ), style=wx.NO_BORDER ), flag=wx.ALIGN_RIGHT, border=self.project.theme.paragraph_space ) add_button.Bind(wx.EVT_BUTTON, self._on_add_button) def _on_add_button(self, event): self.project.add_paragraph(self.page_id)
Setting a minimus size on the page panel is necessary becuase size calculations does not seem to always be correct in a scrolling container.
class PageDropPoint(object): def __init__(self, divider, page_id, next_paragraph_id): self.divider = divider self.page_id = page_id self.next_paragraph_id = next_paragraph_id def y_distance_to(self, y): return abs(self.divider.Position.y + self.divider.Size[1]/2 - y) def Show(self): self.divider.Show() def Hide(self): self.divider.Hide()
def FindClosestDropPoint(self, screen_pos): client_pos = (client_x, client_y) = self.ScreenToClient(screen_pos) if self.HitTest(client_pos) == wx.HT_WINDOW_INSIDE: return min_or_none( self.drop_points, key=lambda drop_point: drop_point.y_distance_to(client_y) )
def min_or_none(items, key): if not items: return None return min(items, key=key)
The title widget is an editable that displays the page title in bigger text.
class Title(Editable): def __init__(self, parent, project, page, selection): self.page = page self.selection = selection Editable.__init__(self, parent, project) def CreateView(self): self.Font = create_font(**self.project.theme.title_font) view = TokenView( self, self.project, [Token(self.page.title)], max_width=self.project.theme.page_body_width ) view.SetToolTipString(self.page.full_title) MouseEventHelper.bind( [view], double_click=lambda event: self._post_edit_start_from_token_view(event.Position) , right_click=lambda event: SimpleContextMenu.ShowRecursive(self) ) return view def _post_edit_start_from_token_view(self, pos): edge_token = self.view.GetClosestToken(pos) selection = (0, 0) if edge_token is not None: edge, token = edge_token start = token.index if edge < 0: selection = (start, start) elif edge > 0: selection = (start+len(token.text), start+len(token.text)) else: selection = (start, start+len(token.text)) post_edit_start(self.view, selection=selection) def CreateEdit(self, extra): edit = SelectionableTextCtrl( self, style=wx.TE_PROCESS_ENTER, value=self.page.title ) edit.Save = lambda: self.page.set_title(self.edit.Value) edit.SetSelection(*extra["selection"]) return edit
All paragraphs are editables.
This paragraph is of type text.
class Text(ParagraphBase): def CreateView(self): return TextView( self, self.project, [ token.with_extra("text_index", (token.extra["fragment_index"],)) for token in self.paragraph.tokens ], self ) def CreateEdit(self, extra): return TextEdit( self, self.project, self.paragraph, self.view, extra ) def AddContextMenuItems(self, menu): menu.AppendItem( "To quote", lambda: self.paragraph.update({"type": "quote"}) )
The view:
class TextView(TokenView): def __init__(self, parent, project, tokens, base, indented=0): TokenView.__init__( self, parent, project, tokens, line_height=1.2, max_width=project.theme.page_body_width-indented, skip_extra_space=True ) MouseEventHelper.bind( [self], drag=base.DoDragDrop, right_click=lambda event: base.ShowContextMenu(), double_click=lambda event: post_edit_start(self, token=self.GetToken(event.Position)), move=self._on_mouse_move, click=self._on_click ) self.Font = create_font(**project.theme.text_font) self.token = None def _on_mouse_move(self, event): token = self.GetToken(event.Position) if token is not self.token: self.token = token post_hovered_token_changed(self, self.token) def _on_click(self): if self.token is not None: post_token_click(self, self.token)
TokenClick, EVT_TOKEN_CLICK = wx.lib.newevent.NewCommandEvent()
def post_token_click(widget, token): wx.PostEvent(widget, TokenClick(0, widget=widget, token=token))
HoveredTokenChanged, EVT_HOVERED_TOKEN_CHANGED = wx.lib.newevent.NewCommandEvent()
def post_hovered_token_changed(widget, token): wx.PostEvent(widget, HoveredTokenChanged(0, widget=widget, token=token))
The editor:
class TextEdit(MultilineTextCtrl): def __init__(self, parent, project, paragraph, view, extra): MultilineTextCtrl.__init__( self, parent, value=paragraph.text_version, size=(-1, view.Size[1]) ) if extra.get("token", None) is None: self.SetSelection(0, 0) else: index = paragraph.get_text_index(extra["token"].extra["text_index"]) start = index + extra["token"].index end = start + len(extra["token"].text) self.SetSelection(start, end) self.Font = create_font(**project.theme.editor_font) self.project = project self.paragraph = paragraph def Save(self): self.paragraph.text_version = self.Value
A quote paragraph is the same as a text paragraph but the view is rendered indented.
class Quote(Text): INDENT = 20 def CreateView(self): view = HorizontalPanel(self) view.AppendSpace(self.INDENT) self.text_view = view.AppendChild( TextView( view, self.project, [ token.with_extra("text_index", (token.extra["fragment_index"],)) for token in self.paragraph.tokens ], self, indented=self.INDENT ), flag=wx.EXPAND, proportion=1 ) return view def AddContextMenuItems(self, menu): menu.AppendItem( "To text", lambda: self.paragraph.update({"type": "text"}) )
A list paragraph is the same as a text paragraph but renders differently. With bullets and indentation.
class List(ParagraphBase): INDENT = 20 def CreateView(self): view = VerticalPanel(self) view.Font = create_font(**self.project.theme.text_font) self.add_items(view, self.paragraph.children, self.paragraph.child_type) return view def CreateEdit(self, extra): return TextEdit( self, self.project, self.paragraph, self.view, extra ) def add_items(self, view, items, child_type, indicies=[]): for index, item in enumerate(items): inner_sizer = wx.BoxSizer(wx.HORIZONTAL) inner_sizer.Add((self.INDENT*len(indicies), 1)) bullet = self._create_bullet_widget(view, child_type, index) inner_sizer.Add(bullet) inner_sizer.Add( TextView( view, self.project, [ token.with_extra( "text_index", tuple(indicies+[index]+[token.extra["fragment_index"]]) ) for token in item.tokens ], self, indented=self.INDENT*len(indicies)+bullet.GetMinSize()[0] ), proportion=1 ) view.AppendChild(inner_sizer, flag=wx.EXPAND) self.add_items(view, item.children, item.child_type, indicies+[index]) def _create_bullet_widget(self, view, list_type, index): return TokenView( view, self.project, [Token(self._get_bullet_text(list_type, index))] ) def _get_bullet_text(self, list_type, index): if list_type == "ordered": return "{}. ".format(index + 1) else: return u"\u2022 "
class Code(ParagraphBase): def CreateView(self): return CodeView(self, self.project, self.paragraph, self) def CreateEdit(self, extra): return CodeEditor(self, self.project, self.paragraph, self.view, extra) def AddContextMenuItems(self, menu): menu.AppendItem( "Create expanded view", lambda: self.project.add_paragraph( self.page_id, before_id=self.paragraph.next_id, paragraph_dict={ "type": "expanded_code", "code_id": self.paragraph.id, } ) )
class CodeView(VerticalPanel): BORDER = 0 PADDING = 5 def __init__(self, parent, project, paragraph, base): VerticalPanel.__init__(self, parent) self.project = project self.paragraph = paragraph self.base = base self._create_gui() <<CodeView>>
def _create_gui(self): self.Font = create_font(**self.project.theme.code_font) if not self.paragraph.path.is_empty: self.AppendChild( self._create_path(), flag=wx.ALL|wx.EXPAND, border=self.BORDER ) self.AppendChild( self._create_code(), flag=wx.LEFT|wx.BOTTOM|wx.RIGHT|wx.EXPAND, border=self.BORDER ) self.SetBackgroundColour((243, 236, 219))
The path widget:
def _create_path(self): panel = HorizontalPanel(self) panel.SetBackgroundColour((248, 241, 223)) self.path_token_view = panel.AppendChild( TokenView( panel, self.project, self._create_path_tokens(self.paragraph.path), max_width=self.project.theme.page_body_width-2*self.PADDING ), flag=wx.ALL|wx.EXPAND, border=self.PADDING ) self.base.BindMouse( self, [panel], double_click=lambda event: post_edit_start(self, subpath=None) ) self.base.BindMouse( self, [self.path_token_view], move=self._token_move, right_click=self._token_right_click, double_click=self._path_double_click ) return panel def _create_path_tokens(self, path): tokens = [] last_subpath = None for (index, (name, subpath)) in enumerate(path.filepaths): if index > 0: tokens.append(Token( "/", token_type=TokenType.RLiterate.Sep, prev_subpath=last_subpath, next_subpath=subpath )) tokens.append(Token( name, token_type=TokenType.RLiterate.Path, subpath=subpath )) last_subpath = subpath if path.has_both(): tokens.append(Token( " ", token_type=TokenType.RLiterate.Sep, prev_subpath=last_subpath, next_subpath=list(path.chunkpaths)[0][1] )) for (index, (name, subpath)) in enumerate(path.chunkpaths): if index > 0: tokens.append(Token( "/", token_type=TokenType.RLiterate.Sep, prev_subpath=last_subpath, next_subpath=subpath )) tokens.append(Token( name, token_type=TokenType.RLiterate.Chunk, subpath=subpath )) last_subpath = subpath return tokens
The code widget:
def _create_code(self): panel = HorizontalPanel(self) panel.SetBackgroundColour((253, 246, 227)) self.body_token_view = panel.AppendChild( TokenView( panel, self.project, self._highlight_variables(self.paragraph.tokens), max_width=self.project.theme.page_body_width-2*self.PADDING ), flag=wx.ALL|wx.EXPAND, border=self.PADDING, proportion=1 ) self.base.BindMouse( self, [panel], double_click=lambda event: post_edit_start(self, body_token=None) ) self.base.BindMouse( self, [self.body_token_view], move=self._token_move, right_click=self._token_right_click, double_click=self._body_double_click ) return panel def _highlight_variables(self, tokens): def foo(token): if self.project.highlighted_variable is not None and self.project.highlighted_variable == token.extra.get("variable"): return token.with_extra("highlight", True) return token return [foo(token) for token in tokens]
Event handlers:
When double clicking somewhere in the path, we want to send a post event with the subpath that we double clicked, and which edge was clicked. That way the editor can select the correct text.
def _token_move(self, event): token = event.EventObject.GetToken(event.Position) if token is not None and token.extra.get("subpath") is not None: event.EventObject.SetCursor(wx.StockCursor(wx.CURSOR_HAND)) elif token is not None and token.extra.get("variable") is not None: event.EventObject.SetCursor(wx.StockCursor(wx.CURSOR_HAND)) else: event.EventObject.SetDefaultCursor() return CONTINUE_PROCESSING def _token_right_click(self, event): token = event.EventObject.GetToken(event.Position) if token is not None and token.extra.get("subpath") is not None: menu = SimpleContextMenu("Path") menu.AppendItem( "Rename '{}'".format(token.extra["subpath"].last), lambda: show_text_entry( self, title="Rename path", body="Rename '{}'".format(token.extra["subpath"].last), value=token.extra["subpath"].last, ok_fn=lambda value: self.project.rename_path( token.extra["subpath"], value ) ) ) menu.Popup(self) elif token is not None and token.extra.get("variable") is not None: rename_value = self.project.lookup_variable(token.extra["variable"]) or token.extra["variable"] rename_message = "Rename '{}'".format(rename_value) menu = SimpleContextMenu("Variable") menu.AppendItem( rename_message, lambda: show_text_entry( self, title="Rename variable", body=rename_message, value=rename_value, ok_fn=lambda value: self.project.rename_variable( token.extra["variable"], value ) ) ) menu.AppendItem( "Copy id", lambda: set_clipboard_text(token.extra["variable"]) ) menu.AppendItem( "Highlight", lambda: setattr(self.project, "highlighted_variable", token.extra["variable"]) ) menu.AppendSeparator() menu.AppendItem("Usages:", lambda: None) def create_open_page_handler(page): return lambda: self.Parent.Parent.Parent.Parent.Parent.Parent.OpenPage(page.id) for page in self._find_variable_usages(token.extra["variable"]): menu.AppendItem( "{}".format(page.full_title), create_open_page_handler(page) ) menu.AppendSeparator() menu.AppendItem("Possible usages:", lambda: None) for page in self._find_variable_pages(rename_value): menu.AppendItem( "{}".format(page.full_title), create_open_page_handler(page) ) menu.Popup(self) else: return CONTINUE_PROCESSING def _find_variable_pages(self, name): for page in self.project.iter_pages(): if self._page_has_variable(page, name): yield page def _find_variable_usages(self, variable_id): for page in self.project.iter_pages(): if self._page_uses_variable(page, variable_id): yield page def _page_has_variable(self, page, name): pattern = re.compile(r"\b{}\b".format(re.escape(name))) for text_fragment in page.iter_text_fragments(): if text_fragment.type == "code": if pattern.search(text_fragment.text): return True for code_fragment in page.iter_code_fragments(): if code_fragment.type == "code": if pattern.search(code_fragment.text): return True def _page_uses_variable(self, page, variable_id): for text_fragment in page.iter_text_fragments(): if text_fragment.type == "variable" and text_fragment.id == variable_id: return True for code_fragment in page.iter_code_fragments(): if code_fragment.type == "variable" and code_fragment.id == variable_id: return True def _path_double_click(self, event): edge_token = event.EventObject.GetClosestToken(event.Position) if edge_token is None: post_edit_start(self, subpath=None) else: edge, token = edge_token if "subpath" in token.extra: post_edit_start(self, subpath=token.extra["subpath"], edge=edge) elif edge < 0: post_edit_start(self, subpath=token.extra["prev_subpath"], edge=1) else: post_edit_start(self, subpath=token.extra["next_subpath"], edge=-1) def _body_double_click(self, event): post_edit_start( self, body_token=event.EventObject.GetToken(event.Position) )
class CodeEditor(VerticalPanel): def __init__(self, parent, project, paragraph, view, extra): VerticalPanel.__init__(self, parent, size=(-1, max(90, view.Size[1]))) self.project = project self.paragraph = paragraph self.view = view self.extra = extra self._create_gui() self._focus() <<CodeEditor>>
Setting up the gui:
def _create_gui(self): self.Font = create_font(**self.project.theme.editor_font) self.path = self.AppendChild( SelectionableTextCtrl( self, value=self.paragraph.path.text_version ), flag=wx.ALL|wx.EXPAND ) self.text = self.AppendChild( MultilineTextCtrl( self, value=self.paragraph.text_version ), flag=wx.LEFT|wx.BOTTOM|wx.RIGHT|wx.EXPAND, proportion=1 )
Setting focus:
def _focus(self): if "subpath" in self.extra: self._focus_path(self.extra["subpath"], self.extra.get("edge")) elif "body_token" in self.extra: self._focus_body(self.extra["body_token"]) else: self._focus_body(None) def _focus_path(self, subpath, edge): self.path.SetFocus() if subpath is None: end = len(self.path.Value) self.path.SetSelection(end, end) elif edge < 0: self.path.SetSelection(subpath.text_start, subpath.text_start) elif edge > 0: self.path.SetSelection(subpath.text_end, subpath.text_end) else: self.path.SetSelection(subpath.text_start, subpath.text_end) def _focus_body(self, body_token): self.text.SetFocus()
Saving:
def Save(self): with self.paragraph.multi_update(): self.paragraph.path = Path.from_text_version(self.path.Value) self.paragraph.text_version = self.text.Value
Data is defined in Image.
class Image(ParagraphBase): PADDING = 30 def CreateView(self): view = VerticalPanel(self) bitmap = view.AppendChild( wx.StaticBitmap( view, bitmap=base64_to_bitmap( self.paragraph.image_base64, self.project.theme.page_body_width ) ), flag=wx.ALIGN_CENTER ) view.AppendChild( TextView( view, self.project, self.paragraph.tokens, self, indented=2*self.PADDING ), flag=wx.ALIGN_CENTER ) self.BindMouse(view, [view, bitmap]) return view def CreateEdit(self, extra): return ImageEdit( self, self.project, self.paragraph, self.view )
class ImageEdit(VerticalPanel): def __init__(self, parent, project, paragraph, view): VerticalPanel.__init__(self, parent) self.Font = create_font(**project.theme.editor_font) self.project = project self.paragraph = paragraph self.image = self.AppendChild( wx.StaticBitmap( self, bitmap=base64_to_bitmap( paragraph.image_base64, self.project.theme.page_body_width ) ), flag=wx.ALIGN_CENTER ) self.text = self.AppendChild( MultilineTextCtrl(self, value=fragments_to_text(paragraph.fragments)), flag=wx.EXPAND ) paste_button = self.AppendChild( wx.Button(self, label="Paste") ) paste_button.Bind(wx.EVT_BUTTON, self._on_paste) self.image_base64 = None def _on_paste(self, event): image_data = wx.BitmapDataObject() if wx.TheClipboard.Open(): success = wx.TheClipboard.GetData(image_data) wx.TheClipboard.Close() if success: bitmap = image_data.GetBitmap() self.image.SetBitmap(fit_image( wx.ImageFromBitmap(bitmap), self.project.theme.page_body_width ).ConvertToBitmap()) self.image_base64 = bitmap_to_base64(bitmap) self.GetTopLevelParent().ChildReRendered() def Save(self): value = {"fragments": text_to_fragments(self.text.Value)} if self.image_base64: value["image_base64"] = self.image_base64 self.paragraph.update(value)
def base64_to_bitmap(data, max_width): try: image = fit_image(wx.ImageFromStream( StringIO.StringIO(base64.b64decode(data)), wx.BITMAP_TYPE_ANY ), max_width) return image.ConvertToBitmap() except: return wx.ArtProvider.GetBitmap( wx.ART_MISSING_IMAGE, wx.ART_BUTTON, (16, 16) )
def bitmap_to_base64(bitmap): output = StringIO.StringIO() image = wx.ImageFromBitmap(bitmap) image.SaveStream(output, wx.BITMAP_TYPE_PNG) return base64.b64encode(output.getvalue())
def fit_image(image, width): if image.Width <= width: return image factor = float(width) / image.Width return image.Scale( int(image.Width*factor), int(image.Height*factor), wx.IMAGE_QUALITY_HIGH )
class ExpandedCode(ParagraphBase): def CreateView(self): self.Font = create_font(**self.project.theme.code_font) view = VerticalPanel(self) token_view = view.AppendChild( TokenView( view, self.project, self.paragraph.tokens, max_width=self.project.theme.page_body_width ), flag=wx.ALL|wx.EXPAND ) MouseEventHelper.bind( [view, token_view], drag=self.DoDragDrop, right_click=lambda event: self.ShowContextMenu() ) view.SetBackgroundColour((240, 240, 240)) return view
A factory can not be edited, so only the view is provided.
class Factory(ParagraphBase): def CreateView(self): view = VerticalPanel(self) MouseEventHelper.bind( [view], drag=self.DoDragDrop, right_click=lambda event: self.ShowContextMenu() ) view.SetBackgroundColour((240, 240, 240)) view.AppendChild( wx.StaticText(view, label="Factory"), flag=wx.TOP|wx.ALIGN_CENTER, border=self.project.theme.paragraph_space ) self.button_panel = view.AppendChild( HorizontalPanel(view), flag=wx.TOP|wx.ALIGN_CENTER, border=self.project.theme.paragraph_space ) self._add_button("Text", { "type": "text", "fragments": [{"type": "text", "text": "Enter text here..."}], }) self._add_button("Quote", { "type": "quote", "fragments": [{"type": "text", "text": "Enter quote here..."}], }) self._add_button("List", { "type": "list", "child_type": "unordered", "children": [{ "child_type": None, "children": [], "fragments": [{"type": "text", "text": "Enter list item here..."}], }], }) self._add_button("Code", { "type": "code", "filepath": [], "chunkpath": [], "fragments": [{"type": "code", "text": "Enter code here..."}], }) self._add_button("Image", { "type": "image", "fragments": [{"type": "text", "text": "Enter image text here..."}], }) view.AppendSpace(self.project.theme.paragraph_space) return view def _add_button(self, text, value): def click_handler(event): if self.project.active_editor is None: self.paragraph.update(value) else: show_edit_in_progress_error(self) button = self.button_panel.AppendChild( wx.Button(self.button_panel, label=text), flag=wx.ALL, border=2 ) button.Bind(wx.EVT_BUTTON, click_handler)
class ParagraphBase(Editable): def __init__(self, parent, project, page_id, paragraph, selection): self.page_id = page_id self.paragraph = paragraph self.selection = selection Editable.__init__(self, parent, project) def BindMouse(self, widget, controls, **overrides): def create_handler(name, fn): def handler(*args, **kwargs): if name in overrides: if overrides[name](*args, **kwargs) is not CONTINUE_PROCESSING: return fn(*args, **kwargs) return handler handlers = { "double_click": lambda event: post_edit_start(widget), "drag": self.DoDragDrop, "right_click": lambda event: self.ShowContextMenu(), } for key in overrides.keys(): if key in handlers: handlers[key] = create_handler(key, handlers[key]) else: handlers[key] = overrides[key] MouseEventHelper.bind(controls, **handlers) def DoDragDrop(self): data = RliterateDataObject("paragraph", { "page_id": self.page_id, "paragraph_id": self.paragraph.id, }) drag_source = wx.DropSource(self) drag_source.SetData(data) result = drag_source.DoDragDrop(wx.Drag_DefaultMove) <<ParagraphBase>>
def ShowContextMenu(self): SimpleContextMenu.ShowRecursive(self) def CreateContextMenu(self): menu = SimpleContextMenu("Paragraph") menu.AppendItem( "New paragraph before", lambda: self.project.add_paragraph( self.page_id, before_id=self.paragraph.id ) ) menu.AppendItem( "New paragraph after", lambda: self.project.add_paragraph( self.page_id, before_id=self.paragraph.next_id ) ) menu.AppendItem( "Duplicate", lambda: self.paragraph.duplicate() ) if hasattr(self.paragraph, "text_version"): menu.AppendItem( "Edit in gvim", lambda: setattr( self.paragraph, "text_version", edit_in_gvim( self.paragraph.text_version, self.paragraph.filename ) ) ) menu.AppendSeparator() self.AddContextMenuItems(menu) menu.AppendSeparator() menu.AppendItem( "Delete", lambda: self.paragraph.delete() ) return menu def AddContextMenuItems(self, menu): pass
CONTINUE_PROCESSING = object()
def edit_in_gvim(text, filename): with tempfile.NamedTemporaryFile(suffix="-rliterate-external-"+filename) as f: f.write(text) f.flush() p = subprocess.Popen(["gvim", "--nofork", f.name]) while p.poll() is None: wx.Yield() time.sleep(0.1) f.seek(0) return f.read()
Editables are widgets that contain a view and an editor. When the editor is opened, the view is hidden and the editor is shown in its place. The editor is opened by posting an EditStart event from the view. This is typically done from a double click event handler.
This class should be inherited, and the CreateView method should be implemented. CreateEdit must also be inherited if EditStart events are expected.
class Editable(wx.Panel): def __init__(self, parent, project): wx.Panel.__init__(self, parent) self.project = project self.view = self.CreateView() self.sizer = wx.BoxSizer(wx.VERTICAL) self.sizer.Add(self.view, flag=wx.EXPAND, proportion=1) self.SetSizer(self.sizer) self.view.Bind(EVT_EDIT_START, self.OnEditStart) def OnEditStart(self, event): if self.project.active_editor is not None: show_edit_in_progress_error(self) return with flicker_free_drawing(self): self.edit = self.CreateEdit(event.extra) self.edit.SetFocus() self.sizer.Add(self.edit, flag=wx.EXPAND, proportion=1) self.sizer.Hide(self.view) self.GetTopLevelParent().Layout() self.project.active_editor = self def Cancel(self): self.project.active_editor = None <<Editable>>
def Save(self): self.edit.Save() self.project.active_editor = None
EditStart, EVT_EDIT_START = wx.lib.newevent.NewCommandEvent()
def post_edit_start(control, **extra): wx.PostEvent(control, EditStart(0, extra=extra))
def find_first(items, action): for item in items: result = action(item) if result is not None: return result return None
class SettingsDialog(wx.Dialog): def __init__(self, parent, project): wx.Dialog.__init__(self, parent) self.project = project self._init_gui() def _init_gui(self): spin = wx.SpinCtrl( self, value="{}".format(self.project.theme.page_body_width), min=100, max=10000 ) self.Bind( wx.EVT_SPINCTRL, lambda event: setattr( self.project.theme, "page_body_width", event.Value ) )
class RliterateDataObject(wx.CustomDataObject): def __init__(self, kind, json=None): wx.CustomDataObject.__init__(self, "rliterate/{}".format(kind)) if json is not None: self.set_json(json) def set_json(self, data): self.SetData(json.dumps(data)) def get_json(self): return json.loads(self.GetData())
A drop target that can work with windows that supports FindClosestDropPoint.
class DropPointDropTarget(wx.DropTarget): def __init__(self, window, kind): wx.DropTarget.__init__(self) self.window = window self.last_drop_point = None self.rliterate_data = RliterateDataObject(kind) self.DataObject = self.rliterate_data def OnDragOver(self, x, y, defResult): self._hide_last_drop_point() drop_point = self._find_closest_drop_point(x, y) if drop_point is not None and defResult == wx.DragMove: drop_point.Show() self.last_drop_point = drop_point return wx.DragMove return wx.DragNone def OnData(self, x, y, defResult): self._hide_last_drop_point() drop_point = self._find_closest_drop_point(x, y) if drop_point is not None and self.GetData(): wx.CallAfter(self.OnDataDropped, self.rliterate_data.get_json(), drop_point) return defResult def OnLeave(self): self._hide_last_drop_point() def _find_closest_drop_point(self, x, y): return self.window.FindClosestDropPoint( self.window.ClientToScreen((x, y)) ) def _hide_last_drop_point(self): if self.last_drop_point is not None: self.last_drop_point.Hide() self.last_drop_point = None
If wx.CallAfter is not used when calling OnDataDropped, the application will not close propery. Not sure why, but this works for now.
class Divider(VerticalPanel): def __init__(self, parent, padding=0, height=1): VerticalPanel.__init__(self, parent, size=(-1, height+2*padding)) self.line = wx.Panel(self, size=(-1, height)) self.line.SetBackgroundColour((255, 100, 0)) self.line.Hide() self.AppendStretch(1) self.hsizer = self.AppendChild( wx.BoxSizer(wx.HORIZONTAL), flag=wx.EXPAND|wx.RESERVE_SPACE_EVEN_IF_HIDDEN ) self.AppendStretch(1) MouseEventHelper.bind( [self, self.line], right_click=lambda event: SimpleContextMenu.ShowRecursive(self) ) def Show(self, left_space=0): with flicker_free_drawing(self): self.line.Show() self.hsizer.Clear(False) self.hsizer.Add((left_space, 1)) self.hsizer.Add(self.line, flag=wx.EXPAND, proportion=1) self.Layout() def Hide(self): with flicker_free_drawing(self): self.line.Hide() self.Layout()
def open_pages_gui(window, project, *args, **kwargs): try: project.open_pages(*args, **kwargs) except EditInProgress: show_edit_in_progress_error(window)
def show_edit_in_progress_error(window): dialog = wx.MessageDialog( window, "An edit is already in progress.", style=wx.CENTRE|wx.ICON_ERROR|wx.OK ) dialog.ShowModal() dialog.Destroy()
class MouseEventHelper(object): @classmethod def bind(cls, windows, drag=None, click=None, right_click=None, double_click=None, move=None): for window in windows: mouse_event_helper = cls(window) if drag is not None: mouse_event_helper.OnDrag = drag if click is not None: mouse_event_helper.OnClick = click if right_click is not None: mouse_event_helper.OnRightClick = right_click if double_click is not None: mouse_event_helper.OnDoubleClick = double_click if move is not None: mouse_event_helper.OnMove = move def __init__(self, window): self.down_pos = None window.Bind(wx.EVT_LEFT_DOWN, self._on_left_down) window.Bind(wx.EVT_MOTION, self._on_motion) window.Bind(wx.EVT_LEFT_UP, self._on_left_up) window.Bind(wx.EVT_LEFT_DCLICK, self._on_left_dclick) window.Bind(wx.EVT_RIGHT_UP, self._on_right_up) def OnDrag(self): pass def OnClick(self): pass def OnRightClick(self, event): pass def OnDoubleClick(self, event): pass def OnMove(self, event): pass def _on_left_down(self, event): self.down_pos = event.Position def _on_motion(self, event): if self.down_pos is None: self.OnMove(event) if self._should_drag(event.Position): self.down_pos = None self.OnDrag() def _should_drag(self, pos): if self.down_pos is not None: diff = self.down_pos - pos if abs(diff.x) > 2: return True if abs(diff.y) > 2: return True return False def _on_left_up(self, event): if self.down_pos is not None: self.OnClick() self.down_pos = None def _on_left_dclick(self, event): self.OnDoubleClick(event) def _on_right_up(self, event): self.OnRightClick(event)
The token view widget displays tokens with different styles. It does so by drawing text on a DC.
class TokenView(wx.Panel): def __init__(self, parent, project, tokens, **kwargs): wx.Panel.__init__(self, parent) self.project = project self.tokens = tokens self.line_height = kwargs.get("line_height", 1) self.max_width = kwargs.get("max_width", 100) self.skip_extra_space = kwargs.get("skip_extra_space", False) <<TokenView/__init__>> <<TokenView>>
First, the token positions are calculated. For that a DC is needed. But the positions need to be calculated before we can draw on the panel, so a temporary memory DC is created.
self._calculate_token_positions()
def _calculate_token_positions(self): dc = wx.MemoryDC() dc.SetFont(self.GetFont()) dc.SelectObject(wx.EmptyBitmap(1, 1)) self.token_positions = [] x = 0 y = 0 max_x, max_y = dc.GetTextExtent("M") line_height_pixels = int(round(dc.GetTextExtent("M")[1]*self.line_height)) for token in self.tokens: for subtoken in token.split(): if subtoken.is_newline(): x = 0 y += line_height_pixels max_y = max(max_y, y) continue style = self.project.get_style(subtoken.token_type) style.apply_to_wx_dc(dc, self.GetFont()) w, h = dc.GetTextExtent(subtoken.text) if x > 0 and x+w > self.max_width: x = 0 y += line_height_pixels if x == 0 and self.skip_extra_space: if self.token_positions and self.token_positions[-1][0].is_space(): self.token_positions.pop(-1) if subtoken.is_space(): continue self.token_positions.append((subtoken, style, wx.Rect(x, y, w, h))) max_x = max(max_x, x+w) max_y = max(max_y, y+line_height_pixels) x += w self.SetMinSize((max_x, max_y))
Drawing the tokens is just a matter of drawing all tokens at the pre-calculated positions:
self.Bind(wx.EVT_PAINT, self._on_paint)
def _on_paint(self, event): dc = wx.PaintDC(self) for token, style, box in self.token_positions: style.apply_to_wx_dc(dc, self.GetFont(), highlight=token.extra.get("highlight", False)) dc.DrawText(token.text, box.X, box.Y)
The Token class:
class Token(object): SPLIT_PATTERNS = [ re.compile(r"\n"), re.compile(r"\s+"), re.compile(r"\S+"), ] def __init__(self, text, token_type=TokenType.RLiterate, index=0, **extra): self.text = text self.token_type = token_type self.extra = extra self.index = index def with_extra(self, key, value): self.extra[key] = value return self def is_newline(self): return self.text == "\n" def is_space(self): return re.match(r"^\s+$", self.text) def split(self): index = self.index subtokens = [] text = self.text while text: for split_pattern in self.SPLIT_PATTERNS: match = split_pattern.match(text) if match: subtokens.append(self._replace_text(match.group(0), index)) text = text[match.end(0):] index += len(match.group(0)) return subtokens def _replace_text(self, text, index): return Token( text=text, token_type=self.token_type, index=index, **self.extra )
Some mouse move action:
def GetToken(self, position): for token, style, box in self.token_positions: if box.Contains(position): return token
def GetClosestToken(self, position): if len(self.token_positions) == 0: return None tokens_by_y_distance = defaultdict(list) for token, style, box in self.token_positions: if box.Contains(position): tokens_by_y_distance[0] = [(token, box)] break tokens_by_y_distance[ abs(box.Y + int(box.Height / 2) - position.y) ].append((token, box)) closest_token, closest_box = min( tokens_by_y_distance[min(tokens_by_y_distance.keys())], key=lambda (token, box): abs(box.X + int(box.Width / 2) - position.x) ) center = closest_box.X + box.Width / 2.0 left_margin = center - box.Width / 4.0 right_margin = center + box.Width / 4.0 if position.x < left_margin: edge = -1 elif position.x > right_margin: edge = 1 else: edge = 0 return (edge, closest_token)
Cursor utilities:
self._default_cursor = self.GetCursor()
def SetDefaultCursor(self): self.SetCursor(self._default_cursor)
The default scrolling window gives extra space. See https://stackoverflow.com/a/22817659. This custom control solves this problem.
class CompactScrolledWindow(wx.ScrolledWindow): MIN_WIDTH = 200 MIN_HEIGHT = 200 def __init__(self, parent, style=0, size=wx.DefaultSize, step=100): w, h = size size = (max(w, self.MIN_WIDTH), max(h, self.MIN_HEIGHT)) wx.ScrolledWindow.__init__(self, parent, style=style, size=size) self.Size = size if style == wx.HSCROLL: self.SetScrollRate(1, 0) self._calc_scroll_pos = self._calc_scroll_pos_hscroll elif style == wx.VSCROLL: self.SetScrollRate(0, 1) self._calc_scroll_pos = self._calc_scroll_pos_vscroll else: self.SetScrollRate(1, 1) self._calc_scroll_pos = self._calc_scroll_pos_vscroll self.step = step self.Bind(wx.EVT_MOUSEWHEEL, self._on_mousewheel) def _on_mousewheel(self, event): x, y = self.GetViewStart() delta = event.GetWheelRotation() / event.GetWheelDelta() self.Scroll(*self._calc_scroll_pos(x, y, delta)) def _calc_scroll_pos_hscroll(self, x, y, delta): return (x+delta*self.step, y) def _calc_scroll_pos_vscroll(self, x, y, delta): return (x, y-delta*self.step) <<CompactScrolledWindow>>
The minimum size is set to prevent the following error:
(rliterate.py:23983): Gtk-CRITICAL **: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkScrollbar
def ScrollToBeginning(self): self.Scroll(0, 0) def ScrollToEnd(self): self.Scroll(*(self.GetSizer().Size - self.Size))
class VerticalScrolledWindow(CompactScrolledWindow, BoxSizerMixin): def __init__(self, *args, **kwargs): CompactScrolledWindow.__init__(self, *args, **kwargs) BoxSizerMixin.__init__(self, wx.VERTICAL)
class HorizontalScrolledWindow(CompactScrolledWindow, BoxSizerMixin): def __init__(self, *args, **kwargs): CompactScrolledWindow.__init__(self, *args, **kwargs) BoxSizerMixin.__init__(self, wx.HORIZONTAL)
class MultilineTextCtrl(wx.richtext.RichTextCtrl): MIN_HEIGHT = 50 def __init__(self, parent, value, size=wx.DefaultSize): w, h = size size = (w, max(h, self.MIN_HEIGHT)) wx.richtext.RichTextCtrl.__init__( self, parent, style=wx.richtext.RE_MULTILINE|wx.BORDER_SIMPLE, value=value, size=size )
The minimum height is set to prevent the following error:
Gtk-CRITICAL **: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkScrollbar
The regular multiline text control does not seem to support SetSelection very well, so we use the rich text control instead.
This code seems to be needed on Windows only, but does not harm on other platforms. If many layout operations are done, the GUI otherwise becomes slow and flickers.
@contextlib.contextmanager def flicker_free_drawing(widget): widget.Freeze() yield widget.Thaw()
wx.CallAfter seems to be needed for the selection to actually be set. No idea why.
class SelectionableTextCtrl(wx.TextCtrl): def SetSelection(self, start, end): wx.CallAfter(wx.TextCtrl.SetSelection, self, start, end)
def create_font(monospace=False, size=10, bold=False): return wx.Font( size, wx.FONTFAMILY_TELETYPE if monospace else wx.FONTFAMILY_DEFAULT, wx.FONTSTYLE_NORMAL, wx.FONTWEIGHT_BOLD if bold else wx.FONTWEIGHT_NORMAL, False )
def show_text_entry(parent, **kwargs): dialog = wx.TextEntryDialog( parent, message=kwargs.get("body", ""), caption=kwargs.get("title", ""), defaultValue=kwargs.get("value", "") ) if dialog.ShowModal() == wx.ID_OK: kwargs.get("ok_fn", lambda value: None)(dialog.Value) dialog.Destroy()
class BoxSizerMixin(object): def __init__(self, orientation): self.Sizer = wx.BoxSizer(orientation) def AppendChild(self, window, **kwargs): return self.AppendChildWithSizer(window, **kwargs)[0] def AppendChildWithSizer(self, window, **kwargs): return window, self.Sizer.Add(window, **kwargs) def InsertChild(self, position, window, **kwargs): self.Sizer.Insert(position, window, **kwargs) return window def AppendSpace(self, size=0): return SizeWrapper( self.Sizer.Orientation, self.Sizer.Add((0, 0)) ).WithSize(size) def AppendStretch(self, proportion): self.Sizer.AddStretchSpacer(proportion) def RemoveChildren(self): self.Sizer.Clear(True)
class VerticalPanel(wx.Panel, BoxSizerMixin): def __init__(self, parent, **kwargs): wx.Panel.__init__(self, parent, **kwargs) BoxSizerMixin.__init__(self, wx.VERTICAL)
class HorizontalPanel(wx.Panel, BoxSizerMixin): def __init__(self, parent, **kwargs): wx.Panel.__init__(self, parent, **kwargs) BoxSizerMixin.__init__(self, wx.HORIZONTAL)
class SizeWrapper(object): def __init__(self, orientation, sizer_item): self._orientation = orientation self._sizer_item = sizer_item def SetSize(self, size): if self._orientation == wx.HORIZONTAL: self._sizer_item.SetMinSize((size, 1)) else: self._sizer_item.SetMinSize((1, size)) def WithSize(self, size): self.SetSize(size) return self
class SimpleContextMenu(wx.Menu): def __init__(self, name): wx.Menu.__init__(self) self._name = name def AppendItem(self, text, fn): self.Bind( wx.EVT_MENU, lambda event: fn(), self.Append(wx.NewId(), text) ) def AddToMenu(self, menu): menu.AppendSubMenu(self, self._name) def Popup(self, widget): self._append_parents(widget) widget.PopupMenu(self) self.Destroy() def _append_parents(self, widget): parent = widget.Parent sub_menus = [] while parent is not None: if hasattr(parent, "CreateContextMenu"): sub_menus.append(parent.CreateContextMenu()) parent = parent.Parent if sub_menus: self.AppendSeparator() for x in sub_menus: x.AddToMenu(self) @staticmethod def ShowRecursive(widget): while widget is not None: if hasattr(widget, "CreateContextMenu"): widget.CreateContextMenu().Popup(widget) return widget = widget.Parent
class FileGenerator(object): def __init__(self, document): self.document = document self.document.listen(self._generate) def _generate(self): CodeExpander(self.document).generate_files() class CodeExpander(object): def __init__(self, document): self.document = document self._parts = defaultdict(list) self._ids = {} self._collect_parts(self.document.get_page()) def _collect_parts(self, page): for paragraph in page.paragraphs: if paragraph.type == "code": self._parts[( tuple(paragraph.path.filepath), tuple(paragraph.path.chunkpath), )].append(paragraph) self._ids[paragraph.id] = paragraph for child in page.children: self._collect_parts(child) def generate_files(self): for key in self._parts.keys(): filepath = self._get_filepath(key) if filepath: chain = self._expand(self._parts[key]) with open(filepath, "w") as f: f.write("".join(char.value for char in chain)) def expand_id(self, paragraph_id): if paragraph_id in self._ids: return (self._ids[paragraph_id], self._expand([self._ids[paragraph_id]])) else: chain = CharChain() chain.append("Could not find {}".format(paragraph_id)) return (None, chain) def _expand(self, paragraphs): self._tabstops = [] chain = CharChain() self._render(chain, paragraphs) chain.align_tabstops() return chain def _render(self, chain, paragraphs, prefix="", blank_lines_before=0): for index, paragraph in enumerate(paragraphs): if index > 0: self._add_text_to_chain("\n"*blank_lines_before, chain, prefix) for fragment in paragraph.fragments: if fragment.type == "chunk": self._render( chain, self._parts[( tuple(paragraph.path.filepath), tuple(paragraph.path.chunkpath)+tuple(fragment.path) )], prefix=prefix+fragment.prefix, blank_lines_before=fragment.blank_lines_before ) elif fragment.type == "variable": self._add_text_to_chain(fragment.name, chain, prefix) elif fragment.type == "code": self._add_text_to_chain(fragment.text, chain, prefix) elif fragment.type == "tabstop": self._tabstops.append(fragment.index) self._mark_tabstops(chain) if chain.tail is not None and chain.tail.value != "\n": chain.append("\n") def _add_text_to_chain(self, text, chain, prefix): for char in text: if chain.tail is None or (chain.tail.value == "\n" and char != "\n"): chain.append(prefix) self._mark_tabstops(chain) chain.append(char) def _mark_tabstops(self, chain): while self._tabstops: chain.mark_tabstop(self._tabstops.pop()) def _get_filepath(self, key): if len(key[0]) > 0 and len(key[1]) == 0: return os.path.join(*key[0]) else: return None
class HTMLBuilder(object): def __init__(self, document, **options): self.document = document self.parts = [] self.generate_toc = options.get("generate_toc", True) self.toc_max_depth = options.get("toc_max_depth", 3) def build(self): self.page(self.document.get_page()) return "".join(self.parts) def toc(self, root_page, levels_left): if levels_left > 0 and root_page.children: with self.tag("ul", newlines=False): for page in root_page.children: with self.tag("li"): with self.tag("a", args={"href": "#{}".format(page.id)}): self.escaped(page.title) self.toc(page, levels_left - 1) def page(self, page, level=1): with self.tag(self.header_tag(level)): with self.tag("a", newlines=False, args={"name": page.id}): pass self.escaped(page.title) for paragraph in page.paragraphs: { "text": self.paragraph_text, "quote": self.paragraph_quote, "list": self.paragraph_list, "code": self.paragraph_code, "image": self.paragraph_image, "expanded_code": self.paragraph_expanded_code, }.get(paragraph.type, self.paragraph_unknown)(paragraph) if level == 1 and self.generate_toc: self.toc(page, self.toc_max_depth) for child in page.children: self.page(child, level+1) def header_tag(self, level): if level > 6: return "b" else: return "h{}".format(level) def paragraph_text(self, text): with self.tag("p"): self.tokens(text.tokens) def paragraph_quote(self, text): with self.tag("blockquote"): self.tokens(text.tokens) def paragraph_list(self, paragraph): self.list(paragraph) def list(self, a_list): if a_list.children: with self.tag({"ordered": "ol"}.get(a_list.child_type, "ul")): for item in a_list.children: with self.tag("li"): self.tokens(item.tokens) self.list(item) def tokens(self, tokens): for token in tokens: { TokenType.RLiterate.Emphasis: self.token_emphasis, TokenType.RLiterate.Strong: self.token_strong, TokenType.RLiterate.Code: self.token_code, TokenType.RLiterate.Link: self.token_link, TokenType.RLiterate.Reference: self.token_reference, }.get(token.token_type, self.token_default)(token) def token_emphasis(self, token): with self.tag("em", newlines=False): self.escaped(token.text) def token_strong(self, token): with self.tag("strong", newlines=False): self.escaped(token.text) def token_code(self, token): with self.tag("code", newlines=False): self.escaped(token.text) def token_link(self, token): with self.tag("a", args={"href": token.extra["url"]}, newlines=False): self.escaped(token.text) def token_reference(self, token): with self.tag("a", args={"href": "#{}".format(token.extra["page_id"])}, newlines=False): with self.tag("em", newlines=False): self.escaped(token.text) def token_default(self, token): self.escaped(token.text) def paragraph_code(self, code): with self.tag("div", args={"class": "rliterate-code"}): if not code.path.is_empty: with self.tag("div", args={"class": "rliterate-code-header"}): with self.tag("ol", args={"class": "rliterate-code-path"}): for path in code.path.filepath: with self.tag("li", newlines=False): self.escaped(path) for path in code.path.chunkpath: with self.tag("li", newlines=False): with self.tag("span", args={"class": pygments.token.STANDARD_TYPES.get(TokenType.Comment.Preproc, "")}): self.escaped(path) with self.tag("div", args={"class": "rliterate-code-body"}): with self.tag("pre", newlines=False): for token in code.tokens: with self.tag( "span", newlines=False, args={"class": pygments.token.STANDARD_TYPES.get(token.token_type, "")} ): self.escaped(token.text) def paragraph_expanded_code(self, expanded_code): with self.tag("div", args={"class": "rliterate-code-body"}): with self.tag("pre", newlines=False): for token in expanded_code.tokens: with self.tag( "span", newlines=False, args={"class": pygments.token.STANDARD_TYPES.get(token.token_type, "")} ): self.escaped(token.text) def paragraph_image(self, paragraph): with self.tag("div", args={"class": "rliterate-image"}): with self.tag("img", args={ "src": "data:image/png;base64,{}".format(paragraph.image_base64) }): pass with self.tag("div", args={"class": "rliterate-image-text"}): with self.tag("p"): self.tokens(paragraph.tokens) def paragraph_unknown(self, paragraph): with self.tag("p"): self.escaped("Unknown paragraph...") @contextlib.contextmanager def tag(self, tag, newlines=True, args={}): args_string = "" if args: args_string = " " + " ".join("{}=\"{}\"".format(k, v) for k, v in args.items()) self.raw("<{}{}>".format(tag, args_string)) yield self.raw("</{}>".format(tag)) if newlines: self.raw("\n\n") def raw(self, text): self.parts.append(text) def escaped(self, text): self.parts.append(xml.sax.saxutils.escape(text))
This generates a file that is suitable for textual diffing.
class DiffBuilder(object): def __init__(self, document): self.document = document def build(self): self.parts = [] self._foo() return "".join(self.parts) def _foo(self): self.pages = [] self._collect_pages(self.document.get_page()) self._render_pages() def _collect_pages(self, page): self.pages.append(page) for child in page.children: self._collect_pages(child) def _render_pages(self): for page in sorted(self.pages, key=lambda page: page.id): self._write(page.id) self._write(": ") self._write(page.title) for child in page.children: self._write("\n") self._write(" ") self._write(child.id) self._write(": ") self._write(child.title) self._write("\n\n") for paragraph in page.paragraphs: { "text": self._render_text, "quote": self._render_quote, "list": self._render_list, "code": self._render_code, }.get(paragraph.type, self._render_unknown)(paragraph) def _render_text(self, text): self._wrapped_text(text.text_version) def _render_quote(self, paragraph): self._wrapped_text(paragraph.text_version, indent=4) def _render_list(self, paragraph): self._write(paragraph.text_version) def _wrapped_text(self, text, indent=0): current_line = [] for part in text.replace("\n", " ").split(" "): if len(" ".join(current_line)) > 60-indent: self._write(" "*indent+" ".join(current_line)) self._write("\n") current_line = [] if part.strip(): current_line.append(part.strip()) if current_line: self._write(" "*indent+" ".join(current_line)) self._write("\n") self._write("\n") def _render_code(self, code): self._write(code.path.text_version+":\n\n") for line in code.text_version.splitlines(): self._write(" "+line+"\n") self._write("\n") def _render_unknown(self, paragraph): self._write("Unknown type = "+paragraph.type+"\n\n") def _write(self, text): self.parts.append(text)
class Observable(object): def __init__(self): self._notify_count = 0 self._listeners = [] def listen(self, fn): self._listeners.append(fn) def unlisten(self, fn): self._listeners.remove(fn) @contextlib.contextmanager def notify(self): self._notify_count += 1 try: yield finally: self._notify_count -= 1 self._notify() def notify_forwarder(self): return self._notify def _notify(self): if self._notify_count == 0: for fn in self._listeners: fn()
The JsonSettings class provides functionality to read and write settings in JSON format.
set.class JsonSettings(Observable): def __init__(self, settings_dict): Observable.__init__(self) self._settings_dict = settings_dict <<JsonSettings>>
@classmethod def from_file(cls, path): if os.path.exists(path): settings_dict = load_json_from_file(path) else: settings_dict = {} settings = cls(settings_dict) settings.listen(lambda: write_json_to_file( path, settings_dict ) ) return settings
def get(self, path, default=None): keys = path.split(".") return copy.deepcopy( self._dict_at(keys[:-1]).get(keys[-1], default) ) def set(self, path, value): keys = path.split(".") with self.notify(): self._dict_at(keys[:-1], create=True)[keys[-1]] = copy.deepcopy(value) def _dict_at(self, keys, create=False): sub_dict = self._settings_dict while keys: key = keys.pop(0) if key not in sub_dict and create: sub_dict[key] = {} sub_dict = sub_dict.get(key, {}) return sub_dict
@staticmethod def property(path, default=None): return property( fget=lambda self: self.get(path, default), fset=lambda self, value: self.set(path, value) )
def load_json_from_file(path): with open(path, "r") as f: return json.load(f)
def write_json_to_file(path, data): with safely_write_file(path) as f: json.dump( data, f, sort_keys=True, indent=0, separators=(',', ':') )
This functions tries to write safely to a file. The file will either be completely written or not modified at all. It is achieved by first writing to a temporary file and then performing a rename.
@contextlib.contextmanager def safely_write_file(path): with tempfile.NamedTemporaryFile( dir=os.path.dirname(path), prefix=os.path.basename(path) + ".tmp", delete=False ) as tmp: yield tmp os.rename(tmp.name, path)
The History class maintains a list of values. The current value can be changed with the back and forward methods. This is used to implement undo/redo behavior.
class History(object): def __init__(self, initial_value, size=10): self._history = [("", initial_value)] self._history_index = 0 self._new_history_entry = None self._size = size @property def value(self): return self._history[self._history_index][1] @contextlib.contextmanager def new_value(self, name=""): if self._new_history_entry is None: self._new_history_entry = (name, copy.deepcopy(self.value)) yield self._new_history_entry[1] self._history = self._history[:self._history_index+1] self._history.append(self._new_history_entry) self._history = self._history[-self._size:] self._history_index = len(self._history) - 1 self._new_history_entry = None else: yield self._new_history_entry[1] def can_back(self): return self._history_index > 0 def back_name(self): if self.can_back(): return self._history[self._history_index][0] def back(self): if self.can_back(): self._history_index -= 1 def can_forward(self): return self._history_index < (len(self._history) - 1) def forward_name(self): return self._history[self._history_index+1][0] def forward(self): if self.can_forward(): self._history_index += 1
if sys.argv[2:] == ["--html"]: sys.stdout.write(HTMLBuilder(Document.from_file(sys.argv[1])).build()) elif sys.argv[2:] == ["--diff"]: sys.stdout.write(DiffBuilder(Document.from_file(sys.argv[1])).build()) else: app = wx.App() main_frame = MainFrame(Project(sys.argv[1])) main_frame.Show() app.MainLoop()
py.test is used for testing.
@pytest.fixture def tmpfile(tmpdir): return str(tmpdir.join("test.rliterate"))
.PHONY: test test: py.test -vv python rliterate.py rliterate.rliterate --diff > /dev/null python rliterate.py rliterate.rliterate --html > /dev/null
.PHONY: watch-test watch-test: inotifywait -m -e close_write *rliterate.py 2>&1 | while read x; do tput reset && make test; done
from collections import defaultdict import contextlib import json import os import re import subprocess import sys import tempfile import time import uuid import webbrowser import xml.sax.saxutils import StringIO import base64 import copy import pygments.lexers import pygments.token from pygments.token import Token as TokenType import wx import wx.richtext import wx.lib.newevent
import pytest from rliterate import *
This chapter documents some insights I've had during the development of RLiterate.
DoDragDrop must be called from within an event handler. (Not custom event handler, but actually like mouse down?)
Font must be assigned before setting a label, otherwise size calculations will be wrong? Must investigate further.
sizer.AddSpacer adding space in both directions
This is a small test program to experiment with scrollbars.
class Frame(wx.Frame): def __init__(self): wx.Frame.__init__(self, None) sizer = wx.BoxSizer(wx.VERTICAL) self.container = self._create_scroll_window() sizer.Add(wx.Button(self, label="other")) sizer.Add(self.container) self.SetSizerAndFit(sizer) self.Bind(wx.EVT_SIZE, lambda event: self._print_info("resize")) self._print_info("init") def _create_scroll_window(self): container = wx.ScrolledWindow( self, style=wx.HSCROLL, size=(200, 200) ) container.SetScrollRate(1, 0) sizer = wx.BoxSizer(wx.HORIZONTAL) for index in range(30): sizer.Add( wx.Button(container, label="Button {}".format(index)) ) container.SetSizer(sizer) return container def _print_info(self, header): print("{}:".format(header)) print(" Frame size = {}".format(self.Size)) print(" Sizer size = {}".format(self.container.GetSizer().Size)) print("----")
Findings:
SetSizer the container scrolls to beginning when the window is activated.SetSizerAndFit the container behaves more like expected.SetSizerAndFit also seems to be needed for the controls to lay out properly.import wx <<classes>> app = wx.App() frame = Frame() frame.Show() app.MainLoop()
self.timer = wx.Timer(self) self.Bind(wx.EVT_TIMER, self.onTimer) self.timer.Start(1000) def onTimer(self, evt): print 'Focused window:', wx.Window.FindFocus()
Literate programming treats any target programming language as an assembly language. RLiterate is not written in Python. It is written in natural language and uses Python as an assembly language.
If I can not clearly explain a part of RLiterate, I probably don't understand it well enough myslef. I might have written code that does sort of what I want, but I don't fully understand it. The act of writing literate programs forces me to understand everything better.
RLiterate works in modes: view and edit. This is in conflict with "no modes". ProjecturED is more direct. It can show multiple projections of the same data structure, but full direct editing is available in both projections. RLiterate can show two projections (view and edit) but only supports edits in one.
Support for more direct editing might be desierable. But I'm a Vim user, and I think modes are powerful. I need to investigate this area more. Can direct manipulation feel as "precise" as different modes? How can automatic save work if there is no explicit "edit ends"?
PDF by Lary Tesler about modes: A Personal History of Modeless Text Editing and Cut/Copy-Paste .
If modes must be used, it should be visually clearly indicated that you are in a mode.
RLiterate started out as a prototype to learn what the program wanted to be. It proved to be useful, and it sparked new ideas of what it might want to be.
At some point it might not be useful to continue work on RLiterate. It might be more appropriate to do a complete rewrite with the learnings found from the first prototype. Instead of continuing building on the first prototype until it becomes a monster, make a new, better, more beautiful version. And continue doing so for version 2, 3, ... and so on.
Random notes of what I might want to work on in the future.
os.rename does not work if file exists (JSON serialization mechanisms)GetWheelRotation and GetWheelDelta" and ' is replaced when writing code in textboxdefResult is never wx.DragMove[[wxdoc:Sizer]]?<a name=... for all pages and paragraphscopy.deepcopy needed? It should be an implementation detail.OpenPage call in code view context menuwx.PopupWindow be used?ChildReRendered be moved up the stack?Generated from 8161d46d6de53ab8fbb13022e5dc3eb07caf17c8.